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ABSTRACT 

The conference was plannel to appeal to a diverse ^ 
audience and was not centered on a single unifying theme. It ms 
divided into three session?. Session I was comprised of two 
presentations which focused on measurement problems at the preschool, 
early childhood, and preadole scent levels: Heaaurement of Cognitive 
Abilities at the Preschool and Early Childhood Level, by Dorothea A. 
McCarthy; and Prediction of Maladjustive Behavior, by William C. ' 
Kvaraceus, Two papers were presented at session II, exemplifying 
basic and applied research: A Theory of Test Pesponse, by Jane 
Loevinaer: and Measurement and Prediction of Teacher Sf fnctiveness, 
by David C, Ryans. Session ITI was composed of a panel which 
discussed: What Kinds of Tes^s for College Admission and Scholarship 
Programs, by Robert L. Ebel and by Alexander G; Hesaan; Criteria for 
Selecting Tests for Colleqe Admissions and Scholarship Programs, by 
John C. Flanaqan; and The Nature of the Problem of Improving 
Scholarship and College Entrance Examinations, by E. F. Litidquist. 
The. luncheon address was delivered by Henry Chauncey and was 
entitled. Some Observations on Soviet Education. (MH) 
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Foreword 



Al the lwenly-8econd annual Iiivitational Conference on Testing Prob- 
lems, some 600 participants considered perhaps as wide a variety of 
topics as has ever been covered at one of these meetings. Thti presenta- 
tions ranged over diflerent facets of measurement from preschool age 
through adulthood, fiom rese*»rch in psychological theory to m thod- 
ology in attacking the problem of delinquency, frohi questions on the 
measurement of teach^'r effectiveness to questions about tht kinds of 
tests used in college admissions and scholarship programs. Taken as a 
whole, the papers reflected the tremendous scope of the measurement 
field and its relation to some of the most pressing educational and social 
questions of our time. 

To fashion such diversified fare into an interesting, cohesive program 
is no small challenge, and we arc all indebted to Roger T, Lennon, 
Chairman of the 1958 Invitational Conference, for meeting that chal- 
lenge so successfully. Dr. Lennon, Director of the World Book Com- 
pany's Division of Test Research and Service, is a man who has been 
influential in guiding the development of the testing movement along 
the path of a responsible and progressive approach to the entire field 
of measurement. We are indeed grateful to him for drawing on his 
knowledge of the field to produce such a stimulating program. 

We are equally indebted to the principal speakers and discussants 
for presenting such ''food for thought." I have no doubt but what the 
ideas and suggestions for future study presented at this Conference 
will l)e much discussed in educational and measurement circles from 
now on, and will, eventually, lead to fresh, productive approaches 
to both old and new prohlems. 

Henry Chauwcey 
President 



Preface 

The annual Invitational Confrrf nee on Testing Problems has achieved a 
reputation as perhapn the outstanding meeting of the year for leaders in 
the testing fieM. With eai h passing year the consistent excellence of the 
programs has attracted a steadily growing audience. As the number of 
tlnme attending has increased, so also has the range of their special 
interest!) within the measurement field broadened; and the task of de- 
veUiping a program of high appi^al to the diversifu d intci chts represented 
in the audience has l:>ecome increasingly demanding on the Chairman. 

Theorist and practitioner, tet^t producer and test consumer^ personality 
researcher and guidance counselor, state testing director and university 
instructor all bring their varied needs and interests and backgrounds 
to the Conference, each confult^ntly expecting a program |hat will be 
uniquely appealing and satisfying to him; and rarely have they been 
disappointed. The 1958 Conference was planned to be of broad and 
diverse appeal, to match fn range of content the heterogeneity of the 
audience ev en to the alninduning of the tradition of a unifying "theme'' 
for the meeting. 

The optMiiiig session of the 1958 (A»nference comprised two papers 
which focu!«ed attention on measurement problems at the preschool, 
early childhotKl and preadob scent level — an area which has received 
{K^rhaps sonh*wbat less than its proper share of research attention in 
recent years. i)cvcUipnients and problems in the measurement of co(r* 
nitive abilities at this level were clearly and comprehensively reviewed 
by Dr. Dorothea McCarthy. Dr. William Kvaraceus spoke on the meas- 
urement an<l prediction of maladjustive behavior, presenting i penetrat- 
ing analysis of the c(»nceptual and methodological problems in the 
appraisal of certain |M«rsonal and social characteristics. 

In ihe second session, papers by Dr. Jane Lm^vinger and Dr. David C. 
Kyars represented neat exemplifications of Imsic and applied researeli, 
tlie former directed to the elaboration iif a new construct in personality 
measurement, and tlic latter to a review of certain findings of the massive 
Teacher Characteristics Study which Dr. Kyans has been directing for 
sevt*ral years. Dr. Koevinger presenlefl a cbisely reasoned argument for 
the existence of a fundamental personality variable underlying responses 
to many personality measures. Dr. Ryans not only presented major 
substantive findings of the Teacher Characteristics Study with respect 
to measurement and prediction of teacher efTectiveness, but also pro* 
jvided a critique (»f mcthodob»gy in this area. Dr. David Tiedeman 



prrsenttMl u Krirf tlint unsioit of tho Loevitigrr thfsis, and Dr. Harry 
Ciltirrt invitiilattt rition income of tin* practical implications of Dr. Ryans* 
work for proM( niH of teacher recruitment, selection, and merit rating. 

At tho tuncfo on meeting Dr. Henry Chauncey presented an absorbing 
account of hin ;)l^8ervations about Soviet education, baaed on his tour of 
Sovii i l(\is>i>i in rariy l%8. Dr. (^haunrey*s report that educational and 
|)sy( hwlo^iral inrasort*rn(*nt as we know it in virtually non-existent in the 
s»'htM)ls of Ku^sia in but «)ne illustration of the mony provocative elc- 
niiMits in ii(*st ri(»tii)n oi an educational system so diflerent from our 
o^n in pliihHuphy, antl practice, and yet so full of significance for our 
WAy nf life. 

lUr afti^rrioon scssitni brought together Drs, Robert L. Ebel, John C. 
Hana/.:.ni, K. K, I.indi|ui>l, and Alexander C. Wesmnn in a panel dis- 
cussion (»n the tiipiit ''What Kinds of Tests for College Admissions?" 
riicsi* aullioriiieH, ronstituting perhaps as expert a panel on this problem 
ar< n.ighl he a^.-^endjh'd, surprised the audience with their diverse, not to 
say conflicting, views. Their lively presentations illumined an area which 
is a*^Huming ever f;realt*r importance in our education. 

It in a pleasure for me to record here my great appreciation to4he 
participants in the l^r>H i lonferenee for their uniformly fine contributions, 
and to K«locali*>nal Testing Service which played the role of host organi- 
zation with it^ ai'i uslomcii grai iousness and eflieiency, I trust that I 
exprc'^s th<* cunstwi-^us of the ()(K) or more leaders in measurement who 
^ attcndcti \\u' r«)rderen( (> in voicing the belief that the 1958 Conference 
diit not tall diort of the high standards set in previous meetings. 

Roger T. Lennon 
Chaiman 



ERIC 



8 



Contents \ 

Foreword by Henry ('jiauncey 3 

IH*eface by Roger T. I.ennon r> 

Session I 

HemnrkM of the ('liairnmn, Kogf.r T. I.knnon, Dinrtur, Divi^it^ti of 
Tejil Research and Service, Worlil Book Company ^> 

MeH8iireinent of Cognitive Abilitii^A at the Pri's^'hool and Marly 
ChildhoiKl Level» Dorotiika A. McCarthy, Fonlliam Univei>ily. 10 

PriHUdion of Mahuijustive Behavior, Wiuiam C. Kvarac kus, Bos- 
ion University and Director Juvenile l)ehn(|neney F'rojeot, ^'a- 
tional t^lueution Asi^oeiation 2u 

Retnark^^ of tlie Chairman 35 

A Theory of Test HesiX)nHr. Jam: I.okvingkk, Jewish llospitul of 
St. Loui^ 36 

Discussion, David V. Tiedeman, Harvard University 47 

Comment, Jane Loevinger 51 

MeaAiiremenl and Prediction of Teacher KITertiveness, Davu* G. 
Ryanji, Uiiiversily of Texaj* 52 

DiifUiiMion, Harry B. Cilrkht, New York Cily Board of Kducatii)n. . 67 



Liimhwn Vddrosa 

Some Observations on Soviet Kduention, Henry ('iiauncey, Educa- 
tional Testing Service 71 



ERIC 



,9 



SosHion 111 



ERIC 



lu Hiarkn of llio t'.lmirnian B7 

WliJit Kiiuls of IVsU fur Colloge AdmisHion and S^liularship 
Programs? Kohkmt L. Khfu Eduralional Tcnjling Service H8 

( .ritma ft)r S'liM-liii^ TeslH forColh^ge Adinisaiotmand S(7lioiarHhi|) 
Pnigrams, John (I. I'Yana(;\n, Hiiivrr^ity nf Piltshiirgh and the 
\iiieriran In^lilulr lor l\r>t irrl» 

riio Nature of the Problem of ImprtJvinK Sliolai-jiliip and College 
Kutraiiee Kxainiiuitioiis. K. F. LiNUyiiisT, Stale University of Iowa 104 

What Kinds of Tests lor College Admission and Seholarship 
Programs? Ai.fa nnokk Wksma.n, Thr INyehoIogieaK^orporaticni 114 

Vppendi\ ^21 



Session I 



RemarL^ of the Chairman 

Roger T. Lknnon, Director, Division of Tenl Reseanh and 
Service, World Book (Company 



I nliould like to opni iUia hrsi sf ssioii of the 1958 Invitational Conferener 
on Tenting Prohleinn by voicing sentiments that I am dure you all share 
with me, gcritimcntd of appreciation to Educational Testing Service, 
our host organization, for \U continuing courtesy in making possible 
these meetings that have, over the years, proven so stimulating to all 
of us in the tield of measurement. 

At this first session today, we shall con&ider two differeMt topics, 
each touching on measurement problems in areas where research has 
bgged behind the need for sound measurement techniques. The first 
arra is the measurement (>%ognitive abilities in very young children. 
I ran think of no better person to review for us what has been going on 
in that area, and lo point out the measurement problems and difficulties 
tliere, than Dorothea McCarthy. Her work, and her special research in 
tlir field of langna^t* development, is familiar to all who are interested in 
child development. Therefore, it is with a great deal of pleasure that I 
will present to you the first speaker. Dr. Dorothea A. Mc(]arthy, of 
Kordham University, talking on ^^Measurement of Cognitive Abilities 
nt the Preschool and Early Childhood Level/* 

Immediately followitig Dr. McCarthy*s presentation, we sh^^^ove 
on to consider measurement questions related to the social problem 
that we call juvenile delinquency. Here is an area where, generally 
speaking, measurement people have been conspicuous by their absence^ 
rather than their contribution. An outstanding exception to this general- 
i/ation is the man who will bv* our second speaker, a man who has 
pioneered in introducing good measurement and research techniques 
into the work of preventing and controlling delinquent behavior in 
young people. He is Dr. William C. Kvaraceus, ot Boston University, 
who i^ currently on a ohe-year leave from that institution to direct the 
Natiiinal (^lu' ation Association's Juvenile Delinquency Project. He will 
talk to us this mornifig about tlie "Prediction of Maladjustive Behavior.*' 
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M(»asuivin(Mit o( (loj^iiitive Abilities 

at the Preschool 

and Karly Childhood Level 

DoHoniKA A. M((:\iniiY, Professor of Psychology, Grnduale 
I* Arts aiul Si itMUTS, Fordham University 



KvtMi ihr casual obrrvor of a group of young children of preschool age 
in iinmt'dialcly impressed with marked individual differences in the 
' \h havifpr nl rhihlreii of the same age and with the very marked contrasts 
ill llu- In hu>ior of children only six months or a year apart in chrono- 
lo^iral age. In motor performances, which it is possible vo observe most 
t»l»jn lively, older children are able to do things fasttjr, more smoothly 
and with grralcr ea^e and precision than younger children. They also 
nianitcst greater strength and are able to attempt ^d perform more 
nnnplex la4s. Thcsic lliingj* hold true whether the obsei ver is concerned 
vsith grv>ss mi)lor prrforir.ancos or with fine muscular coordination. 

rurnin^^ lo the v>lhcr broad area of observable behavior, the verbal or 
linpu-tie, lhrtuii;h ^^hich the child is able to give some reflection of his 
t (»nnpl formatiDn and the higher thought processes, it is clear tlrat older 
rlulihvii lalk more, know more words, and put them together in longer 
and laor.' complex gronpj^ than do the younger children. In their handling 
of words and numbers, o^der children show their ability to be more 
clear and speeilie in contrast to the vagueness which characterizes the 
expresMon of younger children. Also evident is the increasing ability 
of older rhildren to handle abstract ideas, in contrast to the concrete 
iileas \s\\\r\\ are hpieal of younger chiKlren. The degree of complexity of 
tile abslrarl iileas they are able to use increases wi»h age, and they also 
!ihi>w increasing; mental alertness and speed in their ability to solve 
prijbli'ins Nvhii h most p.\>ple would agree are of an intellectual or 
c'fij^rutive iialure. 

[Uvsr are si.ine of the qnalilies of the elTcctiveness of mental function- 
ing whieh llr* lay person refers to when he sayft that a young child 
"Ikh a good IhMd on Ins shoulders," that '1ie will go far," or that he is 
"brij;hl lor hi^ age." The niarketl ehang;^s which occur over a span rf a 
few nn>nth^, and llie sharp contrasts in the mental performancea of 
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DOROTHEA A. MrCAHTHY 

iliffrrnu rluMrtMi of iht* hame ago, srrve lo lugMiik^hl llir rnoinjiiy of 
tlio imliviiluul dilFrrr:* ihi* p yrhologisl alU'inpts to ninifturi;, Tlie«i 
are rJUMTiiilly uoleworlhy ckiriiig \\\v \nvM\\)o\ period, lor it has often 
Intii nhowiv that iiulividual JifTerrnres <4n« any fuftrtioiiare always moHt 
marked durii>(( periods of rapid development. All those evidem e8 of 
menial grovAb are oeeurriiig ii^exU^'^^^^'X coniph*x orpaiiisms whaare 
growing pliysirally in ViKyiiig enViroiimeiitai eireumslaii(*es, and adapting 
ti) varying nuinl>erH and kinds of per^oiii^, l)olh rhildreii and adults. TheHC 
proeesMi;'* of Jiieulal growth are also oerurring in' ctiildren -who are 
.rlinging or independent, withdrawing or aggre^>ive, patient or Qighty, 
mid'Vo on for an infinite variety of pernonality traits w hieh are more or 
le^s elosely (elated to aspeets of mental development w hich we have made 
the mo^t HUJ eessful attem|4s Ut mei'^^ure. 

Mrrely heranse we ran think of iiitelleetnal funrtions in the ahstraet 
and try to ii»4\H^re thern in isolation diit^s not mean that they (MM'ur in 
isolation in natiile^ any morttthuh the ehetnij^t who isolate s iron it: the 
luhoratorf 'tnid^ it in pure form in nature. Just as most eheinieal elements 
an*HKnnti in A;(>mpnuiids of varying degrees of eomplexity,,do, too, eliil- 
drtMi's miiuls are *levef*>ping in youngsters wlo) are reetiving a wealth o{ 
wAvm, aflet^tionate iiurlUrninr; in those wIh» are l»)iiely, deprived and 
Ofgleeled; in tln>se who )H;e thwarted, punished anil reji^ ted at every 
Imn; as wt^ll as in those whoN^e frightmed and shocked into eotnplete 
witldhawal hy tlit»ir early infaiitit?H*x(/erieneert» ^ 

iKnet (the father of the mental testing niovenn tU, on whose ideas we 
Me Htill riahorhting) wiis well aware of nio' lM>f tlie^r things, ;ind he gave 

a tool wliirlu when ada()t»Ml in this roiinlry l)y TeJi^nKin^ geiiiu8» has 
proveii to <»e our lust yardstick for eliildreii of selioif)! age for a whole 
generatiitii. This tt)ol, in «*piti' of itf| many limitalifnis and shorteonitn^^ 
and I'urreiit ohsolesi eiiee^ poiiiteif tlie way for the ikvw w idely aceepteu 
gnuip testing movement iit all edueational levels, 

Whilr psychologists and statisti(|ia'ns have heeii pushing mass testing 
and lioding hetter, more reliahh* methods of testing' large groups with 
innlti|>le.rlioii'e items which can he entered on answer shoels and scored 
hv machine lor easy and (»(licifMit reporting to trhnol administrators, 
huh* prugre-ss Ikh heeri nuide in developing lnols for younger agr levels 
whi( h still need individual examination. 

There is at the present time a tremendous rierti ft)r new tools which 
will do at the infanland preschool levels for today's generation what the 
Stanford'Dinet did fnr the last g.meration of children at school age. The 
increased birthrate since World War It has produced a humper crop of 
pre«cliool children known ns the **l)ahy hf)om/* there is a large prr^ 
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/<>.)« Ini iUitiotud Conference on Testing Problems 

. riitug*- 1.1 tlx- |...|.nlaliu )w in tin- up- l.ra. ket which rrquirw individiicl 

iiiriital I'XumiiiuiiMii. I»rev. nlivr w.uk mid early diagnosis of,' and planning 
for, o|i[)ru|.riutr .Mlu. ati.Mi i^f lianilii a|.|.r.J rliil.iren creatrs a demand for 
pgd and. rvrn highly s,.,M iah«-a tuul.. Thorr arc more handicap^ 
chiKhon who survive nowadays lh<in in formrr times, due to improved 
oh8U-lrii:s an.l |M-dialrirs. Thr fiU'A\ wave of inlrrcRt in mentally reUrded 
.hildren, dur to th.- lunditip log.-lh. r of ih^ir parentu to *hare fheir 
l ummon prohli nis and th.- appropriation of hiige »um8,of money for the 
study, larf and rdurution t f l!:. m.-ntally retarded ha« caught the p»y 
ehoIo«ii al prof.-rotion nhort handi d with only outmoded tools. It is as if 
we wore tryinn to plongh a Held with a horse instead of a tractor. 

As knowli-ilgr ahout what eunslitutoH normal child development has 
spread, deviation^; from what is .-Misidered normal are bringing children 
to the attention ol rhild lauidan-.r rliiiicn and othef service agencies at 
rarlier ageV linlurtunutely. too. oin; niltnr.- has produced increared 
Mumhery of illogitimate rhildren i.i need of placement for adoption at 
eurly ages. Many i liildless eoupl. s are eager to adopt these children, but 
Ihcy have eume to expert some sort of assurance of normality in the 
( liildren they propose to adopt, and hope for ri usonahic accuracy of the 
pirdi. tions that aiV made. Siiire so many studies have shown the 
value of early plnrrnimt in a funiily setting and the harmful effects on 
mental growth of lilV in ati iii>*litulion, an.l mental tests have not proven 
vrry effei live in iiifanry. pl.-irrmrnt agencies have recently been en- 
coiiraging adoption even without henelit of tests. 

:rherr are st-veral reasons whirh seem to have accounted for the neglect 
of this area in the field of psychometrics. The training institutions have 
been preoccupied with training clinicians for the Vetcranf Adminis- 
tration and for work with adults in mental hospitals, and give their 
trainees only a minimum of opportunity to work with children. Child 
development research centers have become preoccupied with longitudinal 
folb w-iip studies which are just now beginning to emerge. After the 
di.Happointiiig results of s..me of the long-term prediction studies, interest 
has turned, among researeh workers,' to deeper and more penetrating 
studies of children's thinking and reasoning; to studies of their concepts 
of causal relationships, of space, time, and number. Usually these 
processes arc studied in a small sample of available children rather than 
in representative samples. Sucli qualitative studies, many of them 
suggested by the very provocative work of Piaget, have not aimed to 
develop prutical tests or measuring instruments. However, they should 
prove highly suggestive to the constructors of tests of the future. For 
example, Harrison (IR) reports a correlation of .7 l)ctween mcnUl 
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aiHr ai)tl rliil(lirii% (MWiiprrhfifriiui nf sstmU iuvnlviiif; \\tuv ronreplft. 

Tlio griHip lVstiii^ nioVi'iiitMit niili IIk* n^i^ult&tit liatulliiig orniaftH data 
hart taught .n<iffij) h ;iln»ul imrrn^, Mn' irn|M»rlaiirt» largo numhers tbo 
torhiiifpioH i\( stnttihiMi Haiuplin^. ittMii analysis, w*'ightiiig and roliahility, 
st» that (Uir staiidanN iis in w h.il t oiisliluto ailtM|ualr \umt\ii havt' afivafUTd 
iTiiiarkahly .imiI havr l"n (»riu' so high tliai il i*^ m» InngiT frasihlo f«»r an 
individual t(» d«'\rlH|» ii \s\\'u \\ \\'\\\ v^ithstalnl tlir cniiral HiTUtiny of 
his ri>llragurH and giiiii \\i(lr^|H(M(l am^ptanro without Huhf^tanlial 
linaiK iai aj*sistiUirr. 

I'sually innitiil i^-^ting dnnr hy \>nnuMi whognirrally find it rasirr 
than nuMi to rrLit* , »nng rfiiMim and tt» rstahlisli rapport with them. 
Many mm ^onu t 't ink it a throat \o thoir nuiHiulinity t(» work with 
young ohiMuMi, ; nd swu. gro^tor intrrrst in things niathomatiral 
and statistir il, d llu ii • \urv rroinunir n'sponsihilitirs, thoy life 
Uiiially divoitrd • \\\r Si group tisting which yioMs greater 
linaneial r- «' inU. \\'i»rnr/i, sm »ho otfirr hand, oftni marry and d(» n<»t 
romain in i'' * lioM or, i* ihi y ihi oontinue activo profoHsionally, ihey 
usually arr in . *.Mr j^rnrlt H nr in traihing ratluT than in research 
^ settings whorr l nMi:l!r havr opportunitlt^s to deveh»p new tools. 

Young ehihlrrn aw* e^-pi i ially halllinganti tliwarting r<» the iieientific 
n'st-arrh wotkeri* wln» inn^t have infinite patience in ordt^r to work with 
ihrm. Thry sirep a t<uge pi>rtii»n of the working day ami woe-betide the 
investigator \sho^e work rooflii ts with a pn^sehool (^hildV regular nap* 
tiinr! Thi'ii. tt»o, yoinig rhiMren suffer from many eontagiou» diseases 
wliiih iVe^prcntly ona-ioii broken nppointmeiits and b»ss of time. 
Preschool chilih fMi ratuiot rorne for cxaminatitnm ab rif and munt alwayn 
be accoinp.o\it'd ti> a ti>ting t^Miter by a mother or other adult wh«» 
may havt* otlu r respi»osiliilitics and idlcn finds it diffieult t«» cooperate. 

Kven whro •-ucli practical dil]\cnltii*s an* overcome, presclmol children 
are notorious lor their sh\ne^s and negtitivism at tTrtain ages, »o that 
llie irst iidinini-lrator, alte' invi*sting an ap(»reeiable amount of time in a 
l a-^e, may IhnI hi* has an incomplcti* te^t bei*ause (»f refused items. In 
adiiition, picsrluM»l children further frustrate examiners with their 
fleeting attention spans and llu ir vivid imaginations which often confuse 
tact and fantasy. Knrthermore, ihey eatni(»t reat! t»r write or even mark 
an an*»wcr hJumM for machine s<*oring. St», il is little wonder they have 
bi^cn iu*glcclcd for easier and niort* lucrative (iebls. 

rhcre is alsiJ the ever-present ticklish problem of validity. What 
constitutes intelligent behavior from age two to age five? Usually we have 
been content with tt sts that show developmental changes with ehrono- 
b»gical agi\ While tliis is a necessary and important characteristic oft 
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trHt for young ehiKiiv;^. many Uiicr things, likr height and weight, also 
HhowinrremrntH with age and ohviounly are not the kinds of things we 
wish to measure wlien we are Htudying cognitive processes. In the pre- 
Mchool field we Jo not even liavo age grade progress or school grades or 
teachers* entiniatrs of intelligence or achievement test scores as criteria 
against which to check our inHtrumentA« and much further work needs to 
he done along tliene linen. Ah Landreth (21) points out, **no attempt has 
HO fur been niaile to relate representative popular judgments of be- 
havior rapai ilieH of young chiMien to their performance on 'intelligence* 
tests.** (p. :YX\) I 

Many predirtivi* studir;^ have1^*n conducted over the years and 
althoHfj^h tliere have heen minor variations due to different sampling 
mi'thfuN, the uhv of tlifferent teHt»> different examiners, and different 
intiTval.H over wliirli the prediction has been attempted, the predictive 
value of all the ti^sts seems to Ih* inversely proportional to the age at 
whit li the first test \^ given and to the interval between tests. Maurer 
(2K) states: ''the studies that have been made show a disappointing 
lark of positive correlation between early str.nding and later standing 
when the intiTvals bi^tween tests are long enough for such informa- 
tion to be useful/* (p. 20) Summarizing the literature, in 1949, Bayley 
(1) said: ''The results of these studies are interesting but have not so far 
given us any adeijuately predictive batteries of tests,** and with regard 
to her own data she saiil: "In all of the comparisons so far made on 
the Berkeley Growth Study children, little consistency in relative'licores 
could be found during the finU two to four years. After this age, how- 
ever, intellectual progress becomes fairly stable." (p. 168) 

The most recent of the/longitudinal investigations is a comprehensive 
study from the FeU Ht search Institute by Sontag, Baker and Nelson (30) 
involving retests on 1 10 ebililren with the Stanford Binet scales from 
ages three to twelve. The authors conclude: **Tbi^ata descriptive of the 
performance^ of the entire group was much like data of other com- 
parable lonf:itudinal studies, . . . The pattern of retest correlation of IQ*s 
at one age with iQ's nt later ages was similar to the pattern found in 
other studies in the literature. Correlations decreased as the age interval 
between the two tests was lengthened and increased as the child grew 
older, if the interval between the two tests was held constant. However, 
the inter-age O(irrclations in the preschool period were slightly higher 
than those previously reported in the literature, suggesting the possibility 
that the correlations during the preschool period may have been some* 
what underestimated in the past.** These authors used a technique of 
smoothed trend lines to minimiie the effect of errors of measurement 
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found in any one lei^l. They report tliul ''the preftchool leftU were found 
to he only aUghlly more variuhle about thi» trend line than the tC8t« 
adminifilered during the elementary icheK)! years/* (p. 52) 

Confirming what Baylcy (4) has called "lability'' of test scores, the 
Pels study (30) reports on the "idiosyncratic nature of the patterns of 
change** which wire found. "It would appear that the extent of IQ 
change found dui ing cliildhood has been previously underestimated. (The 
median amount of change in this instance was 17.9 IQ points.) Sixty-two 
per cent of the children changed m«)re than 15 IQ points sometime 
during the course of mental development from the age of 3 to age 10** 
(pp. 53-5^1). Anothr. inlcrenting rinding was thai icrliiin chddren seem 
io have accel ralive t»r decelerative tentlcncies to llieir mental growth 
curves that appear to be quite independent of special aliilities in types of 
tents parsed. 

I^M^ause of the failure of long-term prediction of infant tests, many 
petjpic have lentletl to di?trard all prnuhool tests. This seems to he quite 
unfortunate, ft>r there are WA» which have gotid reliability and do give 
reasonably accurate predictitHis nftcr al»out two years of age. There is a 
marked change in the testability of children after the onset of speech and 
Ootidenough and Maurer (14) found that verbal and nonverbal abilities 
are readily differentiable on a fairly permanent basis as early as three 
years t»f age. Luulrcth (24) cites H(m3!ik*» (21) Htudy to the effect that 
by ftuir years of age chiblrt n's test scores correlate to the extent of 
about .() with the sct>res tliey earn at six years «»f age. Thus, it is possible 
in nursery schotil io predict fairly accurately which children will be 
rcjily fi>r tirst grade at age six, and to aiti in their school placement well 
enough to avoid some seritms misplacements and perhaps to avoid many 
n( the tragic ca^es of reading disability which emerge from the early 
grailcs anil graduate to tmr reformatories before adolescence. 

So far, the best tools for the measurement of mental functioning have 
been the si» callcd tests of general intclligenre whicli measure only 
abstrart vi-rbal ability. The reason we consider them "best'* is because 
they di» a fairly satisfactory job, better than any other tools do, of 
predicting academic success. We have compulsory c<* ication laws wh*ch 
lorce children into acafl nnic sityatious which trailiiionally have Used 
highly verbal tcchniijur^A. Fo succeed in srheiol, rhildren must achieve 
early mastery of the language arts of listening, speaking, reading and 
writing, as well a« spelling and some facility in dealing with numbers, 
rhrse stM'alled tool subjects umst be mastered if the child IS to be able 
to i-tutly content sobjects and learn something of his cultural heritage. 
This situation has tcndeil to make us keep a narrow focus in our in- 
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trlliKriKT trstiiig. One ul tlir uiaiii tilings y^\\\K\\ prevenU correlations 
lr«)in going niut li higher ih tliut niuny chihJrcMi who have abilities along 
nonverbal lint^s lio not ha\r opportunity U\ nhow them in the traditional 
M»'hnnl or to n^Hri't thmi in ai*liiev<»ni(*nt lent sroret* or other criteria* 
Ah 11 an «U(*h inilivi(lu«iU urr rHeantnl from the pre*ftauretf of a verbal 
ai atleniir iiituation anti have opportunitien to learn traded and earn their 
way in the praetieul sehtud ol experiem r they di> mueh better than we 
Hould have priMlieted on the ba^^in of tlieir early Heoreo on verbal teMa* 

Intelligenee, a.H ^e use the term in pnyi lionictrii H, ih, after all, an - 
ubstraet ti^rni. We lan keep it narrow in focun if we wish to make pre- 
dietioHH to a fairly narrow or Hpecihe criterion such an academic succesi. 
However, riiornilike (:) i) long ago t>(^tTit(Hl out that ahntraet intelligence 
wa» iMily «»ne aspiM t and that there were other typeH ^ intelligence 
which he ealh^l i-om reli intclligeni-e, or lh«» al»ilily to ileal with things 
(perhaps onr pcrft»rniance le?*ts arc getting at thiK), and alHO social 
iiUt»inf»ence, i»r llie ab lity W tieal with people, whii h we have largely 
ignt»i*ed in tlie t.-^ld of nieasnrenient. 

' Am onr knnwiiMlge of «»hild di»velopnient beconie» liroader and richer 
and ileeper, ln»wever, wf can broaden onr e«»ncept of intelligence and 
try li» prediel in «»lher anh^ of hilily tlian the strictly verbal. Gesell 
ptHnttMl tin* way in liis «li»velopmenlal j^ch«*dules which recognized the 
four dimen^ifuis t>f locomotor, adaptive, linguistic and personaI*social 
lM lKivit>r on. an inluilivi* basis, but lie lost the precision «»f his earefu) 
obsi rvalions antI c«>ntn>l of comlilions in his overall Hul»jective appraisal 
f»f the devcbn>ini iilal qiioti/nt and the W\ of statistical equnralence 
of liis variouH srab***. 

IVsls yii»lding a prijfile of snbHcores would be most helpful for purposes 
of differcnlial diagimsin. These can be developed either on an intuitive 
ba>iH or h\ nutans t»f factor analysis. Thurstone (35) has developed the 
latter Ivp mu^l lully in his rriniary Menial AbHilics Scale, but this is a 
group U A. and ul the eailicsl age b^vel with which we are concerned the 
sub^h^sls do n«»l po^.Hr4^ nalisfactory ri»lial»ilily. 

Kulh (JriHillis (Ifi) in her volume Ah\l\tie% af Ikbien described, in 
lorvt, an interesting scale for the first two years t»f life wliich yields a 
prohle of live sul»sct»res as well as an overall (general Quotient. The 
subscales developeil on an intuitive basis are the Lnct»motor Scale, the 
IVrsiMial-Si»cial St ale, the Hearing and Speet h Si ale, one for Eye and 
Hand l>eveb»pnienl, and a Performance St ale. She presents strikingly 
different profiles shtiwing the relalivi^ly b»w performance of deaf babies 
on the Hearing and Speei li seale, of blind ( hihlreo on the Kye*Hand 
scale, and of spaslii infants on the Locomotor scab*. 
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yUiy* WhX lutsril oil 7)71 rliihiirti in KiigLuul whoHf fallit;r«'' ooi upa* 
iiuiiM n>rn'H|HUui roughly lo Uu* iliitriliutioii of thr MiniU'Hota Scale of 
4u i u|>atiiMit» ilrvrloptHi i»y Cuiulriiougli lor tile 19^M) U*S. C.M)»us d«ta« 
It h not litiown, huwrvrr, how rfprrHetitativf they are of British children 
(If how Britinh and Ameriraii rhildren whose fathem have the same 
itrri^KitioHH ciimpare on intelligence. This scale has not as yet been 
widrly U!Mm) m thiii ruuntry. The author is extending it upward into the 
lirrHclHiol and early childhood range. It seems that the greatet^t con* 
Inliution it makes as an infant tH*ale is the introduction of a Speech and 
Hearing scale, an area which previouH infant tests have largely ignored 
or HampliMl very sfHiringly. 

Ah the writer has pointed out rUewhere (26), most scales at the infant 
l< vrl have liecn largely scnnori-motor in characte:s probably because these 
a^pt rtH of lM*havi(»r arc in the ai^cendancy at the earliest levels and arc 
most readily olmervable without instrumentation or highly specialized 
training. They have, for the most part, ignored any attempt to study the 
developmiMit of the precursor^^ of the verbal factor in the early pre* 
linguintie babblings of the infant. The remarkable studies of Irwin (22), 
however, have shown quite clearly that phonemic analyses of these 
r;(rly utterutierm do yield developmental trends which ir many ways 
beliavt' in nun b the same way as do our verbal tests at higher age levels. 
They dilTerentiate lN*tween children living in family and in institution 
settings, even in the lir^t 1^ix months of life. One of my students, Regina 
h-^iebelli (II), largely eonrirnied Irwin's findings on 100 infants between 
(i and IK mos. of age. Sulmetpiently, when Catalano and I (9) followed up 
iipprojtinialt'ly one tliird of her group at an average age of 41 months 
with the Stanfoid.Uinet Form L, we found correlations of .5 between 
c ertain m^aHiires of infant vocalizations su^h as consonant vowel ratio 
and llker Slanford Hiiu^ These substantial positive correlations 
Here obtaitieti with u sample w bieh represented otdy the lower end of the 
iliHtribnlion and were maintained even when ages at time of both tests 
were held eon^tant. 

Althougii our initial attempts to measure mental abdity at the infant 
level did not prove as fruitful as we had hoped, it does not mean that it 
cannot be done with improved tiM*hniques. In this area of measurement 
we an* in a *«tage analogous to the period wln*n James McKcen Cattell, 
who was the lirnt to use the term mental te<)t, was discouri«ged berau.;e 
itis attempts to prediet tbi* aeademie aebievrment of (Columbia students 
i^ith a battery involving strengtb of grip and speed of color naming 
I '-is atid the like proved to be a disappointment. Wc must try again 
with improv(*ii techniques. 
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ThiH is being done by two of the best qualified and most careful 
rcHearch workers I kno^. Narfcy Bayley, who developed the California 
First Year Mental Scale and who has done more longitudinal follow*up 
work than any one else, is now working on a new infaqt scale. Dr* 
Katharine Mdurer Cobb has recently returned from South Africa* where 
she examined large numbers of primitive infants, and is now gathering 
data on an American sample. I have high hopes that these new scales 
being developed by experienced research workers will give us much 
improved tools for the next generation of babies. 

As for the preschool and early childhood levels, I myself am working 
on a new battery of tests which I hope will eventuate in a new point 
scale with a profile of several subscores appraising various aspecfk of the 
child's development. A preliminary pilot study on approximately 100 
mentally retarded children between the ages of 6 and 14 years yielded 
good age trends and differentiated well between educable and non« 
educable institutionalized children. Further data on normal children 
are now being analysed and it is hoped that before too long a full-scale 
standardization with stratified sampling will begin with the items which 
are mo$\ promising, through the gracious assistance of the Psycljio- 
logical (*orporation. 

One of the major problems with mental tests is the problem of 
obsolescence. This matter of becoming outdated affects not only mater iaisi 
but also content and norms. Most examiners are aware, when using the 
Stanford Hinet, of the obsolescent upri^^ht telephones, the black stoves 
with top ovens, the high laced shoes and the steam locomotives which 
many preschool children have never seen'. But perhaps we do not stop 
to realize that in this day of miracle drugs people rarely die from the flu, 
and besides we now call it a virus. Terman and Merrill could not have 
known that sulfa would be discovered the year thefr 1937 revision was 
pnhli.nhed and thus invalidate one of their favorite verbal absurdities. In 
thisiLiy of numerous telephones and automobiles it is indeed rare to see 
a uniiormni messenger boy delivering a telegram by bicycle, and with the 
advent of automatic dryers the sight of clothes drying on a line, as occurs 
in two Stanford Binet pictures, is far less familiar than it used to be. 

I su«4pert» alno, that our norms are quite obsolete, for today's children 
are being exposed to a much more varied and stimulating environment 
tlian the children of twenty years ago. The invention of plastics has 
mihle the manufacture of cheap toys in greater variety possible. And the 
higher standard of living means that more children possess a greater 
varirty oi toy^. Witness, too, the growth of toy stores and the nursery 
i'nrniture industry. 
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The fai't that chililren travel in cars througliout infancy and preschool 
life mraii.H thdr gn^graphical orientation must be greatly expanded over 
that of children of a generation ago who scarcely left their own yards or 
their own block until school age. The rapid spread of nursery schools^ 
has also stimulated more preschool children and probably accelerated 
their mental and social-growth. 

Ju^t as anthropometric studies have shown that children tend to be 
.taller and heavier than they were a generation ago, so psychological 
norms need revisinn and updating from time to time. The recent mono* 
graph by Tomplin (32) which gathered data on children's language 
development, using the same recording techniques and the same sampling 
techni(|ucH in the same city where I worked twenty-five years before, 
showed children using on the average one more word per st tence, 
equal to almost a yearns acceleration^ and showing corresponding 
advances in other aspects of language development. Undoubtedly any 
verhal test would reflect similar up-grading in children's performance, 
due to tlie iiiflu(*nce of radio, television, better standards of living, more 
leisure time which parents spend with children, less use of illiterate 
nursemuids, h*ss bilingualidm and grt^ater permissiveness in dealing with 
children of toiby. It is probable that obsolescence of materials which 
makcH 4ej*ls tnore difficult for children counteracts the effect of their 
greater facility in language development in the total test score^ so that 
we are lulled into complacent continued acceptance of our old 
instruments by an artificial stability in mean score for groups. 

AiMither tendency which I think is unfortunate for the whole field of 
meusuremcnt is that with the development of highly specialized clinics 
for various types of eases in rehabilitation centers, speech and^earing 
clinics and eeirbrfll paUy centers, workers not highly trained in psycho* 
m<*trics are l>eing forced to develop their own batteries of tests and are 
using groups of items which are not standardized and on which they 
dev<*lop their own subjective norms on the basis of their clinical ex- 
perience. We in the field of measurement and psychometrics probably 
would not like toVll these instruments tests at all, but they are being 
UAcd to give diagnoles on large numbers of cases in the service agencies 
b(u\iUHe we have lagg(*d behind and have not sjuppUed the kinds of tools 
which are needeil for the urgent problems of differential diagnosis. 
Ingenious scales of this sort have been developed by Dc Hirsch (10) 
for children with language disorders and by Haeussermann (17) for 
cerebral palsied children. The problem is: what do normal children do 
with the same tasks? Anyone who works for a period of time with one 
' type of handicapped child is bound to develop a distorted norm biased in 
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favor of surh caseH and while discrimiuationtt [)f degree of defect may 
be made carefullvi the perrt|)ective in relation to the nornml child will 
be lo8t without the u^e of sti^iidardized te}^;^ Imsed on representative 
sainpUug^ 

Hatdwin (3) givena pruvoealive disru^^ion of the problem of determin- 
ing what arc the primary mental alulitieB to he tested. Much of the con* 
IuhIoii in thf literature nteniH lri>ni the fart that certain preconceptions 
iiH to the UAtjJre of intrlligence must go into the preliminary selection of 
IteMM included in any Imttcry t»f tcMts. INir matter *how elaborate the 
siih^^rquent statistirul treatment of scores is, wc can never get obt of a 
factor analynif^ elements that were not present in the battery of tests 
to be»gin with. Factor anut ^is may point out a few interrelationships 
which we were not astute enough to anticipate. Correlations obtained 
between two or more fuetor^^ may be due not to any real relationship 
exi?*ting l^etween the two abilities, but to the fact that the two tests are 
measuring or involve certain common elements in the environment. 
'IVrtts may appear to \\v related to each other because both are related 
to a third clement wbirh may or may not be an ability. It appears that 
llofstaetter*s (20) anulysis of Bayley's 1933 data contributed little that 
had not been deduced earlier from a careful study of the standard, 
deviations and the correlations at successive ages in relation to the 
content of the test. J 

With the many practical difFiculties of locating subjects for indivTciaal 
testing, mentioned earlier, one wonders if the necessary preliminary 
te:<ting in order to form a correlaUonol matrix before setting up a new 
battery will prove feasible. It can work out well in group testing at 
higher levels, as in the work of Guilford in personality testing, but 1 
doulit wli(*ther indiviilual tests for young children will be established on 
the basis of previou>ly determined empirical factors for some, time to 
eome. Factor analysis can always be performed after a battery is prepared 
intuitively, and diiplicaiing or irrelevant material can subsequently he 
discardeii If we dare to try out new ideas we may find out things which 
can be confirmed by statistical analysis but which we might be much 
longer in diseovering by purely empirical mrlhods. 
' 'tiecause there has been so little success in measuring a variety of 
aliilities in infant^, the hypothesis Iuih been auvanced that the various 
mental abilities dilVerentiate with advance in age. The evidence for this 
hypothesis is not entirely clear. It may be that ruditnents of several 
abilities are present early, but 'that our techniques have been too crude 
thus far to isolate them and to measure them for study in infancy. 
Irwin^s (22) (*ontriliution on infant language is a striking example of 
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\um refined Ifchniqurs have opened up an area for investigation in 
infancy which was completely overlooked hef^^re. 

t\om the early day» of testing, when we rherishcd the naive hope 
that mental ienl% might somehow measure native capacity apart from 
environmental influenre^. we have come through the era in which we 
have admitted that we aUayn meaf^ure hereditary factors and environ- 
mental influences interacting, and that all we ever get in an intelligence 
lent is present functional level. We have seen numerous studies which 
have attempted to estimate how much each of these factors contributes, 
and the most recent studies are trying to determine how and why 
certain environmental influences operate the way they do. 

There are many parallels and areas of overlap between the development 
u{ language and the measurement jf intelligence, for, as Baldwin (3) 
states, 'Mn the deve lopment of conceptual thought, language plays a 
very signilirant role. . . . The word as a sign of an object implies his 
(the childV) ability to maintain some sort of mental representation of 
the object, action or situation that the word signifies . . . (and) the word 
for an al)straction is very convenient liccause it gives the abstraction 
a concrete handle, the concept is more easily used because there is a 
word for it." (p. 354) There should l»e little wonder, then, that our best 
mental tests are verbal and that vocabulary tests have proven their 
usefulness time and again. 

Research on language development in children is theretore at the 
very core of measurement of cognitive functions, liarlier investigations 
revealfti that rhildrenV language development varied with paternal 
occupation and that children talked in a more advanced fashion the more 
ciHitact they bad with adults. In fact, there is cctnsiderable evidence ^ 
which seems to indicate that the more intense and ibc more prolonged 
ihc contact with the mother, the more accelerated the language de- 
velopment. 

The amazing findings v»fOrBrodbeck and Irwin (8) indicate that the 
environmental impact on language development is measurable even in 
the first three months of life, where infant speech sounds arc much more 
advanced for children raised in a ncirmal family setting than for in- 
stitution infants. Wyatt (37) has spelled out rather clearly how the 
mechanism of unconscious identification is *'the common denominator of 
many of the behavioral events of interaction between mother and chilcj 
and, in particular, for the facts of mutual imitation, so essential for the 
learning of language.'' 

In this connection the work of Goldfarb (13) is of particular interest, 
for it seems |o point out some of the dynamics involved in the acquisition 
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of language a) the giouiulwork for ronceplual ihoughl wliich try to 
measure niih our mental tests at higher age?*. (M»Mfarh, who compared 
institutionalized children and those who greW up in normal family 
settings, points out that in the family the child is cared for by ''specific 
adults railed parents" who are warm and loving and who minister to the 
child with ''detailed understanding/* and that such contact is continuous 
and affords constant stimulation from the same source. Hence it is 
unamhignous and niMrr reailily coniprchcnsihic than (Iceting attention 
from a variety of niiirsc-^ in an institution. (Joldfarh sa)s: "he (the child) 
receives active cncouriigcniont, for example, to hahhie, make sounds and 
then words. . . . Finally, the child's relationship in his parents includes a 
strong element (»f reciproiation'' Thus, he sees the family as a source of 
tender emotions whicli provides the setting necesi^ary for transfer of 
functions from the parent to the child in tiie process of identification. 
The child who thus t xperiences love, ;4ympathy anrd affection in the 
cradle learns to tru^t those ahout liini; he learns lo wait, and to delay 
ininiediate suli^faclions, and hence develops iniu'r control, planfulness 
and fon^sight, which (loldfufh claims is the liasis for conceptual thouglil 
and cognitive development. Klatskin (23) at Yale has shown that infants 
liiisnl on a flexihle rooniing in arrangement tested consideral»ly higher 
on the (lallell Infant Scale than the normative children of ihe Harvard 
(Innvlh Study who were suhjected lo rigid scheduled with less mothering. 

(Ihildrcn who do not enjoy warmth of affection in family settings are 
psychologically iliprivcd. It is prohaldy impossil>le to separate the 
intellectual from the affective aspects in tli(»se early experiences in 
infancy, hut we prohahly never snc ( eed completely in so doing even at 
higher ages, for intelligence is only one aspect of the total hehavior of 
the organism and (*annot he m(*asnr(Ml (Mitirely in isolation. The clinical 
hleralure is reple te with evidences of intelligence test scores which are 
dcpri'ssed when children arc suffering from anxiety and are raised when 
children are experiencing pcri(»ds of relative calm. While, of course, 
si»me children can go thronirh crises of adjustment apparently without 
having their disturbances rcllccted in le»t scores, life experiences do seem 
relat^'d to lahility of test scores with snflici«*nt frcqnency to deserve much 
mof(^ -serious ccmsideration tlian they usually receive in typical large* 
srale measurement studies. 

Two years ago Nancy Bayley (5) addressed this Conference on the 
shape cf the mental growth curve. She raised the question of what kinds 
of emotional climate are optimal at what ages, and what effects the 
attitudes of responsible adults such as parents and teachers have on 
intelligence. She suggested a few clues fr(»m some preliminary data on 
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inatrnml aUilu^IiH. (Uc sfciniJ part of the FeU study (30) referred to 
eurlicr ufu innis to relate the changes in inteUigence test scores in their 
longitudinal data to personality ratii>g8 of the children. Their hypothettes 
that the children gaining most in ratings would show most favorable 
ratings on personality were confirmed. They state, "A stti% of the various 
modes of personality by which children attempt to gain satisfaction in 
their i*xperirnces appeared lo be of value in predicting IQ change and in 
understanding the nature of accelerated or decelerated mental growth as 
related to personality iaet(»r8. During the preschool years» emotional 
dependence on parents appears to be clearly associated with loss in IQ . . . 
During the eh'mentary school years, a cluster of personality traits with 
tlie need for achievement as a common dimension appears to be closely 
associated with accelerUld or decelerated mental growth patterns . • 
during the pre^rhool ye irs the child who develops modes of behavior 
V rharacterizrd by aggressiveness, self-initiative, and competitiveness is 

laying a basic groundwork for future acceleration in performance on 
mental tasks/* (pp. 117-118) These outhora conclude, then, that children 
who pained in 1(J during the preschool years were those who were 
''venturing out of the maternal fold/* 

lMildliirl» (13) lites Howlliy\s (7) cxrellent summary of work on 
it stjtnti(«iuiliz(>d ehildren ulio, lacking maternal stimulation, aflfection 
and support, wi re ri'tardetl mtt'lh elually and "distinctly impaired in con- 
erptnal al ility.** The inipairnient in categoric behavior noted among 
institution rbildrrn was oonsidtTed lo be more than a reflection of low 
inti iligence. " rhi re si i nud to be a lark of difl'erentiation and develop- 
ment of all aspects of personality. Most noteworthy was a generalized 
state of intellectual and emotional improvement and passivity. Along with 
the cognitive disuMlity there were distinct emotional trends; chiefly, 
the absi nre of nornml ciipaeity for inhibition. The institution group 
shoHcd extremely dillleult beliavior with symptoms of hyperactivity, 
rf-itlrs>orss, inaMlily to cunrentrate a»ul unnmnageobility. Further, 
altbnuf;h indiMTiininatin};!) and insatiiiMy demanding of afl'eetion, they 
had no grnuine altai lnnents. Tbry wnc incapable of reciprocating 
tender fn lin^ . . . there was an absence of normal anxiety following 
aggressive or irurl bt huvior (iind) . . . specific impairment in social 
n^aturity.'' 

1*hi.^ syndrome is in marked agreement with Lauretta Bender^s (6) 
desrriptinn of "Psyrbupatliic behavior in childhood'* which she char* 
iK'trri/es bv *'ix\\ itu«!>ility to love or fn*l guilty. There is no conscience. . , . 
rin le is iin inability to eonct ptnali/e, particularly significant in regard 
to time, rbey have no enneept of time »o that they cannot recall past 
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expmiMh r ami laiiiiut t»**;ietit from pohi ex^rieiiiT or be motivated to 
future goaln/* 1 his ijesrriptiuii calls vividly to mind Termatrs (35) early 
description ot nhat constitutes intrlligeiit behaviiir, in which he said it 
iH the abihty to abstract out of pa»t experience thotie essential features 
needed io meet ne\^ ^ituaticwi^^ und the ability to adapt them itt new nitu- 
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Prediction of Maladjustive Behavior 



William C. Kvaraceus, Dire( tc)r Juvenile Delinquency Project, 
National Education AsHociation* 



Juvenile lielinqut^ricy a complex aiiil conttMitious topic constantly 
fermenting on the American scene. Facetl with the dehiuiuency problem 
at the local level, a community eun hope to aeliic'vc »ome measure of 
prevention and control only to the extent to which its efforts are 
characterized by the following: 

1. A positive community attitude. The delinquent mu8t be viewed as a 
child needing understanding and help rather than punishment and 
placement outride the community. It 13 generally true that the de- 
linquent in a hostile child, I at one who also faces an equally hostile 
community. We can't help the delinquent or his family doing business 
on a two-way street «>t hate and hostility. 

2. A knowledge of the delinquency phenomtnon. Tlie community must 
plan on the bai^is of plaui^ible and rcMrarch-oriented theory of delinquency 
as a form of adjustment in our culture and subcultures; it must have 
some knov.ledge of the geography, psychology, and sociology of the 
delinquency act on the local ftcenc; and it must come into the more 
intimate knowledge of the specific .delin(]uent act through case-study 
approaches. Lacking knowledge at these three levels, the community 
may booby-trap itself into •'impracticabpracticaP* approaches (curfew, 
wood*shed, anti*parent legislation) which are irrelevant, if not harmful, 
to delinquency control and prevention. Common sense opinions cannot 
be trusted in this field. We must look for, and stick to, the data available 
from within the behavioral sciences— even in the face of the irrational 
(unscientific) lay critic, the frightening and shrill cry of the feature 
writer, or the crusading editorial commentator. 

3. Early identijication of pre-dclinquent and delinquent. Delinquent 
t>ehavior is not a 24-houv malady; it develops over a long period of time 
and usually with the generous adsistance of two or three adults. The 
future delinquent presents Liany hints and rumblings of his coming 
explosions. 

4. Early referral for study and diagnosis. Once the delinquent or pre- 
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ilrliiiqiient has bfTii identified there must be available ii|)ecial service 
perHofiiiel (psychiilogist, soc ial worker, psychiatrii^t, counselor) to help 
get at the meaning and predisposing factors of the miladapted behavior. 
True, these professional services cost moneyt but this is no dimf-store 
p^)bleni. 

5. Trmtment through coordination of community resources. Using all 
child anil fumily^serving af;tMiricA, the resources of the community roust 
be hrour.ht to bear on the individual in a systematic, scientific, and 
in<)iv iduali/ed cHort of rehal»ilit;itioii leased on case and rommunity study. 

This discu^^ion focn!^(*s on the third aspect, on methodology and 
t(H*hni(|oes aimed to help tlie community worker in the process of early 
identHicatiou o{ tlie pre-dclinquent. Apart frop the primary and direct 
attack on the delinquency problem via the general improvement in 
patternH oi laniily liviii}:, in more effective school programs, in neighbor- 
liood values ntciOi^, and in leisure time ofTerings, etc., delinquency pre- 
vention programs will depend heavily on the ability to identify at an 
early date (perbapt^ as eirly as the (wsi grade level) the youngster who 
in prone, vuhuTable, i vpust d, or suHce{>ti|>le to the delinquent pattern 
of adjiiMluiiMit. 

(Contrary to the iHuai dcpress*cd predictive validity coeflicients reported 
in the literature on forecasting *'occess'or failure in classroom achieve- 
nirnt or on the job, I arn happy to report that it is ( Hsible to predict with 
UM> per rent eHiciency the future delinquents in our society. 

( lunsidering the complexity and the pressures of modern living within 
the' env ( loping web of social taboos, regulations, town bylaws, city 
ordinances, state and federal laws against the fact that there is still so 
nuicb of Adam left in all of us, we can safely predict at least one good 
delinquency (usually more) for efery man. (W itness, if in doubt, tlic 
mores of any out-of-town conventioning groups such as ours.) Everyone 
experiences several delinquencies, officially or unofficially, during the 
growth and maturation process. Even the road to sainthood often seems 
to have been paved witli sins judging from the lives of many of those 
eventually beatilied, lUit all this only raises the crucial question: 
''If horn or u hat are tee forecn^ing?'' I shall readdress myself to this query 
later* First, what are the instruments currently available on the market 
and how efTcctive are they? 

There are at least seven instruments or techniques which arc available 
to the test user and which offer some claim, and sometimes some data to 
warrant mention— if not use— for early identification of the pre- 
delinquent. These include the following: 

Personal Index of Problem Behavior (1) 
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MinneioU Multiphasic Personality luventory (2) 
Porteus Maae Test (3) 

Washburne Social^Adjustment Inventory (4) 
Clueck Predicticn Tables (S) 

Behavior (oirds: A Test lnterview for Delinquent Childl^n (6) 

KD PruneneM Scale and Check List (7) 
None of thcBf items is infallible, nor has any one of these methods 
denionHtrated sufficient forecasting efficiency or power to be used in ^ 
routine or perfunctory fanbion. The best that can he said for some of 
thcni is that they are promising and that they merit perhaps another 
masters, if not doctorate, thesis by way of further or partial validation. 

Kather than review the content of their manuals which are available to 
any discriminating test uner, 1 shall focus on the factors which tend to 
ruise or to lower the reliability and the validity of such instruments. 
First, I shall dincuss the basic premises or assumptions on which pre* 
diction methodology in tlie delinquency field is generally based, although 
not always acknowledged. Second, I will review som^.special construction 
and validation problems that must be solved if delinquency prediction 
is l(» iMH^ome a useful and practical reality, rather than a hopeful research 
fantasy. 



Cuniinuity of hrharior, h\ cluld Rtudy and rehabilitation there is ever 
present tlie backward to earlier life experiences of the subject in an 
ciTort to unlock the meaning of behavior or misbehavior. Stated in 
popular, if not poetic, tongue, past is prologue to the future, the child is 
conceived as father to the man, and concern is expressed with how the 
twigH liavc l>een lient, if not pruned. Prediction assumes a continuity in 
behavior or iniinliebavior, if you will, linked in a cause-effect seqi^ence 
that i.i visible or discernible to an observer, Howwer, the sequitur of 
( aoHe and effect may not be visible to the naked or untutored eye. Most 
observers today view liebavior casually tlirougli the distortion of their 
own bifocal?*, tliiis reflecting the bias of their own theoretical frame of 
reference. The forecaster of delinquency might thus overemphasise or 
underemphayiite data obtained through somatotyping, paycho-genio data, 
psychoanalytic data, or sociological information that might pertain to the 
ultimate effec t as seen in delinquent behavioral adjustment. It is my 
conviction that much of the continuity of cause and effect in delinquency 
in to be ftfund in the cultural and subcultural stream^ rather than 
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rintieildf J among fattorH under the Ain^ although the careful listener 
will note the implied falte dichotomy. Any forecaster today cannot afford 
to overlook the cultural anpects of the delinquency phenomenon. 

If tome dclinquriicy is spontaneous and accidental, this prenrise is 
weakened, or poorly maintained; if sortie youngsters prepare, for their 
delinquencies quietly and pleasantly, as ap|H'ars to be the case in a 
growing numl»er of caHes, the ol>H4TVf*r, whatever his theory, may he 
hard luii to spot the future offender. 

Factor fmHlality, Tilt* f(>ri*<Ni.iter ui^Humes that there is a factor modality 
among rnough variahles that are commonly and peculiarly associated 
with d(*liin|ueiit U^havior. Hiese factor modalities represent significant 
dilTi rifiiri^H'that pile up brtweeii those who become delinquent and those 
wlin do not resort to thin adjustive mechanism. On the other hand the 
Hiiigular and uni(|ur nature of ruch offender's syndrome tends to deny 
and to demnliKh any build up of a useful common modality on which to 
base a forecast of niall>ehavior, At the same time, isolation of a number 
of eommonly observed variables in the backgrounds of delinquents as 
ronti anted with nondeliiiqnent counterfiarts involves an isolation and 
atomizing of elements that saerifices dynamic aspects in causal relation* 
ship, benrr reducing forecasting efficiency, 

Vnrrliah\lit\ of stimulus variables in class and subculture. Whatever 
lurtor, Htiniulu^, i>r variul le is selected for use in a prediction scheme, it 
is lik«'l> to full virtiin to differential interpretation according to the 
iTspoinliMirs \;iliie ^iyslem reflecting the ways of thinking, behaving, 
udjiiHtiiig in tilt* j^nbeiiburt* or class with which he is identified* Hence, 
luve and allertioii ma) be conceived as tender and be identified with a 
hiilabye in the upper cIuhh; b)ve p<*rceived by a hiwer class adolescent 
nuiy be viewed lirrce and vUdent and be identified with a family Tight, 
tliUH proving the worth and importance of the young mcmlior in the 
family arena. A good example ean he found in the use of the affection 
ittMii ill the (flueck ton iul fartiir» ' table with a Maltese father, who cub 
tuially nt*ver displays open aiTet tioii for his young, althoogh the table 
e\|HM tain y in that be should do so. School uebievemcnt may be slurred 
in tbr loHer rla^n home and praised in the up|H*r class family, duplicity 
and cunning may be i^xtolb d in towr class living and deplored in upper 
rlaHc< !i1embrr*«bip. 

If d( liii(|ueiicy is an essential and more typical aspect of life among the 
loHrr elassr<4, these differential responses can be exploited in the fore* 
cutting gnnit\ However, as more delinquents tend to l>e drawn from the 
npper levels of community strurtiire in the futurci these stimuli and 
variable!* y^hivU invite difTereiitial response's among youth may only 
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•uccced in iorling them into lh« cUwet «nd/or lubcultufei from which 
ihf y come, or with which ihey molt easily identify. The value of such 
cU^s identification would thus be lowered for prediction purposes. 

Contingency i(\ pndicting 6eAav>r. Predicting future adjustments or 
maladjustments will always be m-.a* on a contingency basis. If the sub. . 
ject's situation improves, the prediction of delinquency will be weakened; 
if the situation deteriorates, the foiecaster is more Ukely »o ^ "B**' 
this time. Hence the prediction made In time must be vit wed as relative 
to Bul.»equent conditions. Failure to predict accurately may often reflect 
validation of the methodology albeit it depresses the validity coefficient. 

Short-term vs. hng-term prediction. Som'e forecasters, particularly the 
Glufcks, have assayed long-term prediction working with the six-year-old 
i.r from the first grade level. Just as it is haxardous to plan a picnic or a 
Hkiing trip on the basis of l6ng term weather forecasting, one must be 
prepared I't.r ilisappoiatnients as well as surprises. Obviously predicting 
at the junior and senior high school levels, closer to the point of delin- 
quency precipitation, should yield a higher level of validity coefficient 
than in long range forecasting. A pertinent question arising is that of 
adequate time allowance in the validation of any prediction scheme to 
insure adequate measure of delinquenry fall-out. Current British studies 
reported by Mannheim suggest that 18 months may l.c suflieient to check 
the prediction power of some measures with older youth who have been 
institutionalized. Beyond this we are in the dark about what constitutes 
minimal time duration in an effective validation design. 

Such are the major premises on which prediction techniques are set. 
Some of these assumptions may withstand close inspection while others 
cannot be accepted axiomatically. Hence, we do not have a firm bedrock 
on which any prediction scheme can be automatically and easily 
t-ttnstrucSed. 



Spccinl Mt'tliodologicnl Problems 
iit Construction und ValidQtion 

Assuming that a workable base can be squared off, the forecaster must 
still face the following special problems in the test construction and 
validation process. 

Validation design. There is no substitute for the before-and-after 
research design in validation studies , t prediction tools. This means 
that ihe prediction technique mult be applied to a sample of youngsters 
•nd forecaiti made. A reaionable period of time for behavior and mil- 

30 



WIIXIAM C. KVARACtUS 

behavior to take place mutt be allowed during which adjustment criterion 
data must be gathered according to some acceptable definition of mal- 
behavior. Finally, the relationship between the forecast and the be- 
havlbral adjustment must be established and expressed in terms of 
prediction eflficiency. 

Many of the techniques now on the market depend too heavily, er^n 
exclusively, on construct validity or concurrent validity. Many validation 
studies merely telescope tbe before-and^after design by using direct 
comparisuns between available criterion groups and still others attempt 
to validate forecasting effectiveness via retrospective analysiif. Through- 
out many of thr^ie studies there is ^ome confusion between what con-^ 
stitutes probability and what denotes predictive efficiency in a statistical 
design. 

Before-and-after studies are expensive and difficult to manage. Loss of 
cases due to mobility alone is a serious stumbling block to the research in 
such long-term experiments. But there is neither a haven nor excuse in 
these dilBrulties for poorly executed research. 

The criu rion: W ho or u hat is being predicted? To return to the question 
raided earli«*r : "Whom or what are we predicting?'* What kind of criterion 
data are to be collroted on each individual in the post-forecasting situa- 
tion? Who and what is a delinquent? The term, juvenile delinquent, is an 
omnibus concept that can include the large bulk of our youth population 
since everyone can, and does, easily fall by the wayside at one time or 
another in our moil? complex Garden of Eden. 

We muixt tirnt observe that there is no dichotomy between delinquents 
and non-dcliuquents (except m terms of the court tag, but even here the 
dichotomy breaks down as one studies the informal and formal dis- 
positions of cases). The implication is that the statistical design in 
prediction of malbehavior and delinquent behavior is not amenable to 
the expediency of a binerial r. Misbehavior exists on a continuum. What 
the research(*r lacks is a graduated measure of the delinquency phenom* 
enon itn a malbehavior scale. Until such a measure, based on some 
system of habituation and seriousness of offense, is worked out the 
^ unreliability of the criterion measure itself will seriously reduce the 
validity coefficient assuming that a high degree of relationship exists be* 
tween forecasting iechnique and adjustment. 

All existing forecasting devices have attempted to predict to the galaxy 
of any and all kinds of delinquency without due regard for any diagnostic 
differentiation according to modalities (types) of delinquents. This is 
perhaps their greatest defect. Separate validation checks and prediction 
tables n^rd to Ite evolved for the following types or modalities: the 
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nvumtic detifUfaent, hoavy with aiuirty and guilt; the socialiied delinqumt 
who has failed, f(ir good reason, to iiiteriiahze the value syritem of 
duiiiinaiit Horiety and whoxe f^uper-ego in already delinquency identified; 
llie ovei't'iggressive UfuvHvdized delinquent wlume liehavinr rej^rcsents 
a Htrong dt feiine, even olTeime, again^it ^luthority figures conceived as 
liiHlile, threatening, and predatory. To the»e major modalities nil|ht be 
adtled othern, viz, groii|) intonieatcHi type, trnumalized delinquent, con* 
Htilutional ty(H\ and (MThaps others. ' 

Ketiiiemeot in vulidatioii experiment will Hwait refinement in differ- 
ential diugnoHis in the proeeH8 of gathering eriterion data against which 
in test the foreeanting instrument, always working within the rubric of 
eaeh modality. I would hazard the hypothesis that we can succeed in 
predietiiig eertain modalities of delinquents with greater effectiveness 
than othec.H. For example, it may be relatively easy to identify the future 
HO iali/ed and unsoeialized delinquents and relatively difficult to predict 
in the neun>tfe category of delinquents. At the same time it may prove an 
impossible task to predict the traumatized offender because of the 
accidental nature of this phenomenon. 

One additional tiote bears mentioning. The popular practice of re- 
mirling to the use t>f such convenient and undifferentiated criterion 
gnnips as eourt easi»s or, worse, institutionalized delinquents on whom the 
enmrnunity has given up or who have)>een removed f»*om the community 
lor spe( iai reasons, should be erased from validation studies which aim to 
sfi up prediction tables. The special and hardy breed of screened de- 
lintpients obtained through such sampling do not lend themselves to fair 
lest or experimentation. In a sense the cards are stacked in our favor. 
Without the use of any elaborate device most youth workers in any 
eommunity can foretell what youngsters are most likely to be banished 
to the training institutions. 

Need for local Vfdidation, The delinquency problem varies from one 
etimmunity to another and in the large urban centers it v/ill vary frOm 
iu»i^hborhood to neighborhood. Each community or neighborhood will 
show significant variations in incidence, type, and time of misbehavior, 
reflecting unique elements in the pop .lation and in the culture and/or 
subf'ultures. Any prediction tool jbat has been demonstrated as useful 
in a iarg** Nan center with a mixed or heterogeneous population may 
pnwe to W of little or no value in the more homogeneous and mono* 
lithic culture of suburbia. Any promising instrument now available on 
the market needs t<» undergo local * alidation reehecks. This will call for 
considerable research interest, effort, and skill on the part of test users 
at the local level. 
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The iiffil and iiereAnity fur IcH^al validation of prediction tf chniquea iH 
also pointed up by the problems presented in i)xt class structuring of 
American society. Sociological studies infer that the average New England 
community, for example, might need at least three separat^^itions of 
a prediction scale or tuMe: one for the lower class, one for ihe middle 
class, and one for the up|>er c^sas. Rather than hoist six editions to suit 
\Varni^r*s family status studios, I have compressed the categories 
|>erhaps too conve^nicntly. On review I might prefer four editions. As one 
studies the items on some of ;he scales and check lists, the obvious 
irrelevantly of many of the stimuli for children of varying family status 
in our society is such as to render them useless and meaningless. 

ObM'mttiOn r.Y. U\%i situation. In developing a methodology of prediction 
we will neiMi to favor the use of observation techniques such as check lists, 
graphic rating scales, anecdotal records as against the use of test items or 
HclTinventory questionnaires which place a much too heavy burden on 
the reading aMIity, trustworthiness, and seriousness of purpose on the 
part of the respondent. The combination of low reading capacity^ 
irrelevancy of response^ and cultural duplicity of many pre-delinquents 
often tends to lower the reliability of the best of these instruments. 

Furthermore the technique that is evolved must be easily administered 
to large clas«irnom groups. To build prediction tables assuming Rorschach 
testors, P8yrluatrir Interviewers, and trained Social Workers will not 
result in any usable tir practical detection methodology as we con- 
template ten million youngsters in the high schooU of the nation. 
What wc need, for example, is a handy method which trained teachers 
can employ as they come in close and continued contact with their 
student^. * 

Whatever instrument is 'devised, it must face the practical test of 
serving as an improvement over what might be accomplished even now 
through the careful reading of a casc-study folder or a cumul/.tiveTccord 
tile. The professional worker, clinically trained in child development 
and adolescent psychology, can generally smell out a future delinquent 
through a cateful perusal of a child's case *-ecord Prediction methodol- 
ogy mu^t proviilr a shorthand metbcKl that is at h*Rst as effective as 
this longliand approach. 



Summnry Slateinont 

How inu(*h hope can be extended to the comnnniity workers who are 
concerned with prevention and control of malbehavior through early 
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ideniifioalion of iht? potential nonconformer? If wc can strengthen the 
ba«e on which prediction methodology must be built through a careful 
re-analysia of all out major premiftcs, and if we can carry off our con* 
Btruction and validation pro'^aei with the refinements which have been 
indicated, it is likely that we can predict delinquency as well aq tests of 
academic aptitude predict academic achievement— which, as you all 
know, is not very well. Even then, or especially then, we shall need to^ 
recognize that the prediction tool and the data gathered thereby have in 
no way relieved us of the urgently and the neccssitjL of careful and 
deliberate judgment in drawing conclusions, u»ing another available 
inrormatU)n, concerning the cliild^s exposure or proneness to mal* 
behavior and delinquency. 

;. i 
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Remarks of the (ihairman 



III this second Hessiun of the Conference, we shall hear reports that 
difTer appreciably in content and approach from those presented in the 
first session. With diversity a9 our keynote, this is to be expected. 

Those of U9 who have been concerned with test development and te^t 
^istruction are aware of some of the very notable and significant 
c^ributions to test theory that have come from the lady who will 
speak first. She is Dr. Jane Loevinger, of the Jewish Hospital of St. 
Louis, who will discuss "A Theory of Test Response.*^ 

Knowing full well that some of the points Dr. Loevinger raises will 
stimulate a desire fur disc4is»ion— and knowing equally well that diti- 
cu^^sion time is limited— we have resorted to the strategy of asking 
one person to voice his reaction to her thesis. He is Dr. David V. 
Tiedeman, of the School of Eilucation at Harvard University. Speaking 
as an individual, and not in any sense serving as surrogate reacting foi 
all of us, Dr. Tieilcman will discuss Dr. Loevinger's theory immediately 
following her presentation. 

Then, still bound by time limitations, we shall proceed to the nfxi 
topic on our program. Earlier this morning we heard a discussion on 
the prediction of maladjui^tive behavior among children. In this session 
we shall hear a report on research in another important area of pre- 
diction, that of the "Measurement and Prediction of Tea<^her Effective* 
ness." Dr. David G. Ryans, of the University of Te:Kas, is especially well 
qualified to bring us this report. He was Director of the National Teacher 
Examination :4 for a period of seven years, is currently Director of the 
Teacher Characteri.stics Study, and will draw extensively on the findings 
in that investigation in his presentation today. 

Following Dr. Ryans' report, we shall haV^a brief comment from a 
man who it concerned, day in and day out, with precisely this matter of 
selection of teachers and the problem of how to identify good teachers. 
We are fortunatr in having Dr. Harry B. Gilbert, of the Botrd of 
Examiners of the New York City Board of Education^ here today to 
discuss some of the implications of Dr. Ryans* research findings. 
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III recent motithflt working in collaburalion with Professor Abel Ossorio 
and Mrs. Kitty laPerriere, I found taking shape a conceptualization of 
a trait which I currently believe to be the major source of variance in 
structured personality tests, regardless of their intent* Manifestations 
of the trait have been called facade, test-taking defensiveness, response 
set, **fe:ocial desirability," acquiescence, and so on. The term rt^ponse 
bias can serve as generic for these phenomena, though Jackson and 
Messick (12) prefer to emphasize that they are components of personal 
Mtyle. The fact that response bias is a manifestation of the trait by no 
means implies that the trait is inconsequential outside of the testing 
situation. On the contrary, its importance in the test situation reflects 
its importance in many other aspects of life. 

The trait may be defined metaphorically at the ability to usume 
distance from oneiielf, or more exactly, as capacity to conceptualize 
imeself. It is one cognitive aspect of ego development. That it should 
greatly influence the kind of self report which most personality tests call 
for is obvious; not quite so obvious^ is that capacity to conceptualize 
oneself varies as a function of agt\ of education, and of one^s station in 
life. Note that the trait does not refer so much to the content of one*s 
self-concept a3 to one*s ability to form a self-concept. At least three points 
are needed to bring the dimension into focus. At the lowest point there 
is no capacity to conceptualize oneself; at the midpoint there is a stereo- 
typed, usually ronventional and socially Acceptable self-conception; and 
at the higheot point a differentiated and more or less realistic self*concept. 

Let us look first at how this trait normally develops with age. We are all 
familiar with the baby's wonder as he discovers his own body. But at 
stake here is the conception of one^s psychological rather ti.zn one's 
physical person. The moment of discovery may be perhaps the time the 
child first says ^'Bad*' to himself as he does or refrains from doing some- 
thing his parents have proscribed. At that moment the child has con* 
ceptualized himself u having impulses which are sometimes bad and are 

*Pri»parilion of this psper wis iuppjflad by i reteirrh grant, M4213, fron the 
Naltonil Institute of Menlit Health, Public Hftlth Service. 
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not nece88arj|y acted on. That he has achieved a rudimentary idea of good 
and bad in no more important than that he hat a rudimentary idea of 
impuUe and control. Extension and elaboration of the pair of constructs, 
impulse and control, are major tasks all through childhood. A( this. point 
the argument is remini»ront of Ki»lly*s (14) psychology of personal 
constructs: to haVr th«* construct of impulse is to have its opposite, con* 
trol, and thus to achieve a degree of choice. What Kelly does not seem to 
recognize clearly is that in just this instance, not having the construct of 
impulse does not eliminate impuUes from one^s repertory but rather 
leaves onevcomplctely at their mercy. 

By early adolescence the ability to conceptualize one^s impulses and 
the concomitant degree of control is fairly well established. But the 
typical adolescent is in many respects an ^'authoritarian personality" 
(1). He is prone to think in stereotypes, to be punitive, disciplinarian, 
conventional, anti-psychological and intolerant of those who are different 
(8« 18). In terms of tlu* usp<'ct of ego development here being described, 
he has achieved distance from his impulses but not from his ego. He 
has some ability to think about himself as a psychological person, but 
his self-characterization tends to follow a conventional, socially ap- 
proved stereotype. The strongly derogatory 8clf«*portrait which is also 
common in adolescence is equally stereotypt^d. Everyday observation 
leads to the suspicion that the derogatory and flattering stereotypes may 
alternate in some children in short time span. 

During the college years there is in favorable instances a change from 
the typically authoritarian to an intellectually sophisticated point of view. 
There i? usually, or at least oftt*n, a marked increase in capacity to view 
oneself with some detachment, to see onescif as having a style of life, 
to report feelings without taking refuge in conventional stereotypes. 
The chang(*s which uikc place during the college years in personality in 
general and test liehavior in parti;^,ular have been documented in the 
Vassar study of Sunford, Freedman, and Webster (21). That response 
stereotypy on tests is somewhat characteristic of college freshmen but 
not of advan<Mi undergraduates has been noted by Christie, Havel, and 
Seidenberg (4). * *^ 

Thus the capa( ity to attain distance from oneself grows with age, 
front infancy, where there is no distance frpm impulse, through ado- 
lescence, where there is dis^tance from impulse but not ego, to the college 
years, where there is distance from ego as well as impulse. Were the sole 
purpose of this discussion to present a picture of the aormal course of 
e|(o devftOpyietit« this would be a pale and one-dimennional version of 
Erik Eriksoti's (H) viv id and dynamic portrait, "Growth and Crises of the 
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Healthy Per«onalily," prepared for the Midcentury White House Con- 
ference. My purpoie is not to deBcribe personality development but to 
make a contribution t»» personality measurement. Erikson'a brilliant 
paper does not, by itself, lead to measurement. The elision of some of the 
stages Erikson describes results from using available psychometric 
research as a sieve fur bis intuitive observations. 

The usefulness of the concept of egi) development as a psychometric 
dimension depends oi 'whether one can convincingly describe some 
individuals in terms of levels of ego development not characteristic of 
their age. When we speak of an adult as having a mental age of 2 or 8 or 
10, we of course do not mean that his behavior is identical with that of the 
average child of the given age. Rather there is an abstract characteristic 
of his behavior which^n l>e so measured. Similarly, some children and a 
few adults have as little control and as little capacity to conceptualize 
their impulses as infants or small children. Redl and Wineman (17) have 
depicted prca<lolescent children of this type in a book which contributes 
much to our understanding of ego development. Many, perhaps most, 
adults have a self-conception hardly less slereolyjped than that char- 
aclcrlHiio of adolescence. If we lake a slice of the pbpulation of constant 
age, we will fnid ego development as measured on this hypothetical 
scale conrlaled wilh intfUigence, with educational level, and with some 
measures of social class. Since intelligence, social class, and educational 
level are themselves intercorrelaled, this represents a single additional 
datum in support of the conceptualization. But while it is a single 
argument, i' is supported by a large amount of research. Several recent 
summaries nf research with the California F scale, which is as much a 
measure of efto level as of anything, have confirmed these relation- 
ships (3,5,23). 

There have been now presented three lines of argument in support of 
the construct of ego development. It is a constantly increasing function 
of age, at least through the early adult years. It tends to increase con- 
stantly as a function of intelligence, educational level, and social status. 
And it can be conceptualized as increase in a single function, to wit, 
capacity to assume distance from oneself. There is a fourth argument: 
ego development tends to increase constantly wilh psychotherapy. 

.Ml forms of psychotherapy push the patient upward on this dimension. 
In the case of delimpient or disorganized persons who remain at the low 
end of the scale beyond the appropriate age, increase in conventionality 
and control is an aim. In the case of conventional people, increase in 
sophistication, in capacity to conceptualize themselves, is not necetwrily 
an aim but is the means of therapy. Rogers (20), in describing the thera- 
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lunuic proirrih, Iium poHtulutod eHHeiitiully the same dimenHiuii. An 
apparent paradox is that Hogers de^ieribfS the later stages of therapy in 
tcrniH of fircm;iei/ distance fnmi one^s feelings. This p.iradox is explained 
in (Hvchoanalytic writings in terms of a temporary splitting of the ego (7), 
Tn achieve distance from oneself is the condition for achieving immediate 
grasp of feidin^s as ftTlings. The patient must talk about himself and hia 
teelingH in the therapeutic tran«aclion, whirh surely implies a capacity 
to attain distance from them. 

iri I'Mik at characteriMtit* manifestations of different levels of ego 
tl veiopmriil in |K'rs«mality t6st.s. The most striking fact, of course, la the 
tenticnry o{ mtiMl [)*M>plr to answer in trrms of response stereotypes, 
moHi in)tal)ly» a defeiisive or faviaal^lr self -portrayal. The tendency to 
dcscrilM' oneself favorably has In^en shown U) increase between the ages 
nf H and 13 (10) and to decrease during the college years (21). This 
iinu*monot(Hiir relation between "socially desirable*' self-portrayal and 
age correHpoiids exactly to the non-nmnotonic relation between con- 
MMitinn^iJity and ogo develufunent. Thiit is, conventionality tends to 
inrrcase as ne g > from I si to middle level, and to decrease between 
middle and hight^st l(*vel. The non-monotonio rotation )>etween the most 
obvious phi in»\ypic test nianifcNtation aii'd the genotypic trait is surely 
a inajoi ob^^tarle to personality mcaHurement. 

In iniuiHiiriog maladjustment, neuroticism, and the like, one must set 
up a set o( n ?*ponsrH as '*ih»rmal/' INyrhologi«ts in recent years have 
tended to u^r i« statistical drfmitioii of normality. So doing, however, 
Joes not a ti I tlie normal key vot y much from what Wf)uld have been 
rlioHiMi a priori by p?.yclioitigislH of a more naive era, for th * socially 
atreptable rr^pt)n!<es are just the ones chosen most frequently by the 
numerous middle j'roup. Middle-da^.s children and adults tend to appear 
a little l>eltci adjustctl on persofiahty tests tlwn their lower-class con- 
te^i^M)raries (2). On the otiier hand, Vassar seniors tend to test more 
maladjusted than Vassar freshmen (21), Wliile personality changes 
iMnl<.nl>tcillv Vike place dgring the college years, they are probably pre- 
dnminanlly iii the direction id* greater ego development ond intellectual 
maturity. Its seems uidik»'ly that basi. 'Ily the seniors are a lot more 
ii.ahuljustf d tlian tliey Iniii been as freshmen. Rather, they are more self- 
rritieal, le^s < (niventionarand stereotyped in their thinking They are 
ca{)able of admitting to ronneioiisness ^nd to their test responses problems 
which had U^en iln»re pM along but were concealed beneath a facade of \ 
normalitv. Ju-^l thij* s(\/t of phenomenon has made measurement of adjust- 
ment enormnumly diflic^ult, For psychotherapy itself tends to move the 
patient up thr srale of ego development. And, other things equal, greater 
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fgd maturity at the upper extreme leadit to a decrease in stereotyped 
favorable »eirp<>rtrayal. No doubt there are many exceptions to the 
latter generalization. Some sick people give a stereotyped unfavorable 
self*portrayal, and therapy could be expected to brighten their self« 
portrait. Moreover, svmptonis which actually disappear could be expected 
to be so reportcnJ. All in sH, however, evidence indicates that response 
to structured p«*rHonttlity trsts \% more clearly related to ego development 
than to ttdjustmonl. 

What the- behavior is of thone lowMt on ego development is 
not known in drtail. These people include small children, as wdl aa 
older rhildrrn u:vl a(l^llts in whom impuUivity is unduly predominant. 
Anyone who hun tried to obtain tests from individuals of low social status, 
when* the lowest irvel of ego development is overrepresented, has dis* 
covertnl that ret u^alH to cooperate and sabotage of various sorts are more 
frequent than at lii^hrr social levels. The suspicion of and resistance to 
such small authority as a research psychologist represents is itself a fact 
worth recording, and strikingly similar to the negativism of pre-school 
children in testing »itnati.ons. 

These individuals do become acres»ible to psychological observation 
^through more or irss involuntary referral to guidance clinics, alcoholic 
trealnictU centers, and so on. Their inability to put their troubles, 
however overwhelming, into words is only partly a matter of opposition to 
.|tie anthority ol' the clinic. Skillful, sympathetic clinicians report that 
it appeiirs tH represent a genuine inability to conceptualise themselves. 
DilTuse physical ronipiainls seem to represent a kind of "body English,** 
i.r., their p)iysical complaints may represent in part psychological 
i/ialaisr fyr whiclrthey have no concepts. 

Resistance to authority, impulsivity, and lack of ability for self* 
ronceptualization: surely this is a coherent syndrome, and one different 
from the idiMHilicalion with authority which characterises the midpoint 
of our variablt^ Documentalir.i of this syndrome can be found in the 
description of the frtwest social class by Hollingshead and Redlich 
(U. Ch. 4), though I do not maintain that all individuals in the lowest 
class are at the lc»west level of ego development. 

The ideas presented here are meant to apply chiefly to objective per- 
Htmality tests. There is, however, one problem of long standing in 
projective testing, especially llu^ Thematic Apperception Test (TAT), 
which ban some relation to these concepts. The punIc is, when do 
aggressive resp(uiscs on the. TAT indicate aggression in overt behavior, 
and alternatively, when do aggressive fantasies substitute for aggressive 
behavior? Prol^ably no one has a complete and clear-cut answer to this 
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({ui^Htion. Thrrc art* inclicatiotu that in youtliH Trom low lucial clauses 
there is a positive relation bi'twe<*n overt and fantasy agression; in 
higher social classes, a negative relation. The pMyrhoanalytic concept of 
'*primar]r process** helps to hriilge the gap U^twern this finding and the 
ronrept of ego development. Piedoniinance of primary prcM j is a 
translation of the impuUivily \vhii h chararteri/eM the lowest level of c^o 
tievelopment. For imlividuaN ut thin IcvH wonlH and fantasies serve to 
trigger the kinds of liehavior thry symholi/e. Since there is minimal 
control of impulse expri*ssion in l>ehaviQr, the impulse's expressed ia 
fantasy are the same as those expressed in tiehavior. in the two higher 
lovrU of ego development, on the contrary, secondary process is well 
(\Htal)lish(*d; whirh it* to say* words and fantasies serve to delay, control, 
and substitute for expression of impulses in hehavior. Therefore, it is 
not surprising to tind a slight negative relation between fantasy aggression 
and overt aggression in middle and upper rloss groups. 

Lyie and (lilrhrint (15) com|)ared TAT protorols of delinquent boys 
with those of a matched control group; there was no diflerence in the 
numl^er of aggressive or anti-social themes exprissed, hut the non- 
delinquents used varijiius devices to indicate greater distance from anti- 
st»rial impulse s such a?* denial of the reality of the situation, inhibition 
of the impulse by guilt, and rationalization of theitnti-sorialact. Note that 
l.yleand (filrhri.Ht usv the same metaphor, distance, to indicate the means 
by whicli control over impulses is maintained, ond that they find 
reprcHentation of tlic control devices in the T\t protocol. Purcell (16) 
ftunid similar rev^ultn studying psychiatric referrals in an Army training 
camp. He divided his ra«4CH into three groups according \o case history 
evidence of iinti»?*ocial l onduct. Best difTerentiation of the groups was in 
terms of fantasy thcmc^^ of iiitrrnal punishment ami ratings of the 
aggrcs«4ivc fanta>ic»4 ti> "rcnmirncss,** which referred to time, place, 
deforce (d reality, and mi on. The anti-social group nhowed few themes of 
inti^rnal punishment and litth* rcnintcncs*< from tlieir nggresHive fantasies. 
While Purcell interpreted the al)senec of themes of internal pioiishment 
in tlie anti-Koetal grt^up in ?iii|H*rego terms, note tliat it is also evidence 
of lack of ability to conct ptuali/e inner life. 

|)r. Kenneth Isaat^s has ju^^t sent nie a MS. in wliicb he develops a 
eonstrnet very similar tn v^hat I call eg<» development. He calls it 
''relatability/' nlre^^ing the ( apacity to pererive (ither people and capacity 
lor difTerentiated interperhoiial relations, liis studies also show that level 
of relatal>ility can Uv judged from TA T prottM^ols, but specific criteria 
are not listed. 

In sketching ef^o dt^velopmenl as a major (liincnsion of personality. 



II 



1958 Invitational Confemu^ on Testing Problems 

I am Uttiiig B^iiet^i work at my modrh Hi» breakthrough in the field of 
ability rora«uremeiit, whirh haft not been matched in the following SO 
years, lUccerdtni* I briievt% because he found a process which corre* 
Apuiiflefl to an intuitively perceived dimension. There have been attempts 
to deliiK* personality traitH after Hinet's model. But if we imitate him too 
closely, we end up mraHuriiig almost the same trait that he did rather 
than a personality trait. In fart, c(»rrelati(m of a personality test with age 
or intelligence i» often interpreted as that much evidence for invalidity. 
Yet it in absurd to a<^Hert that personality does not change with age, and 
both gratuitous and contrary to everyday observation to assume that per* 
sonality trendn nill be uncoi;relatcd with IQ or social status. We cannot 
lift ourselves out of this problem by our correlational bootstraps; we 
need to mind the psychological content of our measurements. 

The dimennion of ego development 1 have sketched is a kind of 
common denominator in Erikson*s description of the normal process of 
t go development; Rogers* description of the process of therapy; Sullivan, 
(ftant, and Grant'n (22) description of the growth of capacity for intei- 
|H*rsonal relations; and results with objective personality tests pertain* 
ing to authoritarianism and response stereotypy. The three papers de* 
sci ibing process ail refer to seven stages, 'vhether by coincidence or not. 
From the psyrhonietiic pttint of view, each involves a forbidding array 
of details. On tlir other hand, the pHyrliomctric approach has been to 
'assume that everyone can'^be classified as having more or less of some 
one thing, like dominance or adjustment, or lies somewhere between a 
pair of poles, like authoritarian-democratic. The idea that if you can 
name it, you can measure it, dies hard. The journ ds are full of studies 
using little atl hoc trsts of traits that struck that research worker's 
faiiry. 

1 have, tlirn, followed Hinet in using process as touchstone of di- 
mension; but I have tried to avoid the eircumltanlial details of particular 
prorrssr^, as nrll as the noniinalistie fallacy which still vitiates many 
p^ycfionietric appr(»aclies to personality. 

riie authors of The Anlhoritnrinn PtrwHatity (1) considered and 
rejected the idea that the authoritariao Was an immature version of the 
liU*ral person. However^ the meaning of authoritarianism shifted in the 
course of their rcj^ear^ h. At its inception they were concerned witl^a 
harsh and pathological extreme of anti-Semitism andfsscismt which they 
found only in a few indiv iduals in their San Quentm sample, who fit the 
description ut the biwest level of ego d(*velopm^nt. The core of the trait 
which emerged from thrir studies was very much like what I have 
descrilwd as tfie middle stage of ego deveh»pmen;, much less vicious than 
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what they lookinl fur at firttt. The reaaon the California group wu 
diverted away from the political aspects of aulhoritarianism and in the 
direction uf ego development is that the latter aspects are f%t, more 
pervasive in personality and are just the aspects of personality most 
accessible tt) lAcanuremiMit and interview. ^.^^.'.m . 

The California dinrluimer thai authoritarianism and liberalism are 
stages in a developmental process has not stotnl up. Evidence that 
adolescence i^ typically a nu»re authoritarian period than later maturity 
has come from many indc{)cndcnt ^ources^ including clinical observation 
(8), opinion pulU (18), and studies with the F scale. The thinking of 
the Gdifornia group evolved from that of looking for a few wicked 
authoritarians to recognizing the authoritarian tendencies in large groups 
of ordinary i>eople. Th^y never quite admitted that the conventional 
authoritarian represents the norm in our society. To see authoritarian 
tendencies in a developmental framework, as I have tried to do today, is 
to carry the evolution of the concept one painful step further: The* 
struggle against authoritarian tendencies in one which each of us must 
make within himself, and it \% a battle never wholly won. 

That the fight against authoritarianism takes place in each of us was 
the theme of Krirh Fromm*s (0) 1941 book, Escapt from Freedom. But 
Fromm, seeing the similarity in the child's spontaneity and the spon* 
taneity whirh ran (h* recaptured by a truly (nature 'ai^ult» wrote as if 
people knew ihcir ceal selves and th( n deliberately surrendered that 
knowlctlge tn slip into a conformist or authoritarian stereotype. The 
dialectics of growth seem more accurately represented by the sequence: 
impulsivity, rigii contml enforced by intellectual stereotypes, and 
flexible controls ^nforced by genuine insight. Riesman (19), though 
much influenced by Fromm, has drawn a p! iUre essentially the same as 
that sketihed here. His term for the loWe».t i *vel of ego development is 
anomic; for the middle level, conformist or, most often, adjusted; for the 
highest level autonomous. Rir«mnn has enriciied our understanding of 
(lifl'erent patterns of conforming by tiis description of the tradition* 
directed, the inner-directed, and the (»tber.directcd man. These types of 
c onformity characterize the middle {?vel of ego development in different 
societies and in different groups witUn a given society. So far no one has 
tracrd the difTerentiiil manifestations on tests of the inner-directed and 
the other-direrted man, though there has been at least one attempt. But 
Allen Edwards* (6) finding, that the number of people clalmitig that an 
item describes them is a high rectilitiear function of the independently 
judged ''social desirability** of the item is remarkable evidence for our 
ether»dirertednrss. 
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Propone ntA of factor analyait, cluster analyniit and multidimentlonal 
•<'aling set up artificial ptoblenu with boiet or random numbers and 
demoniitrate that their preferred method will indeed capture the 
dimenaiona built into the problem. I have begun instead with a real 
trait of central importance in test behavior and would now emphasise that 
factor analysis or clu<iter analysis or multidimensional scaling could not 
possibly reconstruct such a trait. Impulsivity is a distinguishing mark 
ot the lo irest level of ego de velopmen t, but thi) flexible controls of mature 
life are phenptypically closer to the impulsive stage than are the rigid 
controls of the intermediate stage. The non«monotonic relation between 
conventionality and ^o development has already been noted. The com« 
plex of ego development leaves many traces, and with ftepeet td each qf 
them there are individual differences. Factor analysts make much of 
getting from phenoty}>ic variables to genotypic traits. Bui only such 
genotypic variables as are linearly* or at least monotonically, related to 
phenotypic ones will be revealed by factor analysis. By thMnselveSt 
statistical techniques can yield only partial insights. I trust* however, that 
no one will carry away the mensage that I don*t think it worthwhile to 
manter or use difficult statistics. The psychological research worker 
who does not understand statistical principles is as handicapped as the 
paychometrician who does not permit himself to develop a feeling for the 
traits he studies. Factor analysis is an important technique in the hands 
of a responsible psychologist with insight into the psychological con- 
tent of his variables. 

Suppose you answer that you prefer to stick with whatever factor 
analysis reveals, that you find nothing comf^lling about the construct 
! have sketched. This raises an interesting and profound question, one 
which will be answered neither in short time nor by the self-elected. 
Since personality is complicated enough to encourage many alternative 
constructions, what are the criteria for the validity of alternative ways 
of construing it? Ego development as here sketched provides a framework 
within which one can view such major researches as The Aulhoritarian 
Pertonality and subsequent related studies (1.3,23); Redl and Wineman*s 
(17) TV Aggrt$sive Child; Riesman*s (19) studies of American character; 
Erikson's (8) work on growth and crises of the normal personality; 
studies of personality development in the college years \>j Sanf^rd and 
others (21); Edwards* (6) work on the social desirability variable in 
personality tests; work on the relation between content and style or 
response bias (12); Rogers* (20) study of the process of therapy; and 
Kelly*s (14) psychology of personal constructs. Dotens of smaller or 
less familiar studies contribute also to the overall picture. The line 
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of research which urigintlly gave ri»e to these sper ulalions, work which 
I have been doing with Blanche Sweet, Abel Ossorio, Kitty UPerriofe, 
and others on patterns of child rearing, I have not even mentioned. 
Work now in progress is testing the hypothesis that different patterns 
«»f child reiriiig characterise different levels of ego development; here 
in another far-reaching application. 

If memory serves, factor analysis originally aimed to give an economical 
account of much data i^tth few concepts. I am claiming that the single 
construct I have proposed accounts for much data. By contrast, 
application of factor analysis to personality tests has too often taken 
siiudl amounts of daU and developed a confusingly Jarge number of 
constructs. Has factor analysis of personality tests given rise to any 
powerful constructs, any constructs of sufficient utility, for example, 
that clinicians have made use of them? 

A problem of concern to the Educational Testing Service has been 
meai^uring the behavioral outcomes of higher education. I would like 
finally to show liow this problem is related to the discussion. In regard 
to education at the nursery school and kindergarten level, no doubt 
specific behaviors can be used to measure the success of the educational 
endeavor. The child is taught to lay his coat on the floor, slip his arms into 
it and flip it on by raising his arms. He must learn to conform to bells, 
commands, and classroom routines. The aim of university education is 
emphatically not to inculcate such stereotyped behavior patterns, but to 
free the graduate from conformity to cultural and l)ehavioral stereo* 
types. I do not have any pat suggestions as to how to measure the outcome 
of higher education, but it seemv safe to say that the search for specific 
behavioral outf^omes is doomed to failure. It represents, moreover, a 
spurious and misguided objectivity. William James made the point in his 
essay Oh Harvard: 'The day when Harvard shall ^tamp a iin^e hard and 
fa^l type of character upon her children, will be that of her downfall. 
Our uiulisciplinablfs are our prouflc^t product** (1.3, p. 3SS). 

Sunuuary 

A cognitive asipect of ego development, ability to conceptualire oneself, 
is poHtulateil as accounting for a major portion of the variance in 
structured personality tests. At least three points are needed to define 
the dimension. At the lowest point there is no capacity to conceptualise 
oneself as a psychological person; at the midpohit, a stereotyped 
self conreption: at the highest point, a differentiated, realistic self- 
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eniiceptioii. Mure or lens synoiiymouAly, at the lowest point there is 
no distance from impulses; at the midpoint, distance from impulse but 
not from ego; at the highest point, ability to assume distance from ego as 
well OS from impulse. , This trait increaiies constantly with age; for 
constant age tends to increase with intelligence, education, and social 
status; and tends to increaH^e with psychotherapy. However, ego develop* 
ment has no conspicuous constantly increasing manifestations. Its most 
conspicuous nianifesl^ution in pt*rsonulity tests, tendency to answer in a 
stereotyped, usually a socially approved styl^ is not monotonieolly re- 
lated to the trait, tending to decrease in the upp^r range and»p~nbably 
tending to increa^i^e in the lower range. A further difficulty in mei ring 
favorable outcome of higher education, and incidentofly, favorabl )Ut- 
come of psychotherapy, is that the highest level of ego development is 
charai terized precisely by the absence of stereotyped, objectively 
specifiable U^haviors and attitude I have followed Binet in using process 
as touch^itone for dimension, but not imitated him too closely for feat^of 
returning exactly to general ability. / 

Many methodologists, until recently including me, believe th^t our 
job iH to p^frfect a method fur discovering traits, and the right method 
will lead w straightaway to a complete (catalogue n( important traits* 
Purely for ii^ shock value I nish to rrcord a contrary hypothesis, that 
rvery triajor human truit wilt l>e di^iovenHl and establiHhed by a unique 
method. Whether that hypothesis is true or not, methodological 
Sophistication in the absence of psychological acumen will lead only to 
fragmentary dimensions and insights.' 

KKKF.HtNCKS 

1. AimSNo T. W.« FncNKM.^HsrMHWiic, Emc, I.kvin^o.^, D. J., and SAnroaD, R. N. 
The authoritarian pfru^nality. Nfw York: Harper, 1950. 

2. Aixn, F. Iiiflurnci' ol social tLuk on prrsonalily lr»l rnponKcs. PsychoL Bull*, 

3. CifSiSTiK. R.. ANh Cook. PrcCfY. A fiiMn to publidhnl lilrmture rrlaling to the 
auihoritarian prrtinnalily through 1^56. /. Psyrhitl,, i%B, 45, 17M99. 

I. riisisTU. K., llwr.L, Joan, ahu SRinKMnKSG,' H. I§ Xhn ¥ uralr irrevflrtible? 

ahnorm. P%\chol., I9:>H, 5ft, I H IS'). ' 
;V (loiiM. T. S. Thr relation tif ihr Kdcair to intrlliconcr. /. joc. Psychoit 1957, 
46, 207.217. 

6. FdwaHDk, a. Thf int'int deMrtMlity variable in pfrwnality oi^Msmeni and rtteanh. 
New Vi>rk: Drvdrn, 1957. 

7. F.n^ttiN, R. iWhoanalvlic trrhniqucK. In D. Browrr and L. E. Abt (Edt.), 
Prt»gfts$ in chnival psyckotogy* Nfw York: Griinf & Sirallon, 19S6. Pp. 79«8/. 

8. F.RiaiON. K. II. Growth and criiea of th<* hralthy pfraonabtv. In M. J. E. Senn 
(Ed.), S>mposium on thi healihy p9r$onality. New York: Joaian Macy, Jr. Founda- 
tion, 1950 / 

9. FaoilM« EmcH. t$cape from frttdom. New York: Far -ftr & Rinehart« 1941. 

10. Gtniu« J. W., a!«d WAUn. J. J. The method of paired direct and proieetiire 
<iueitionnair«a in the atudv of attitude structure and iocialiaation. Piychol, 
ilonogr., |95a, 72 (I). 



'16 



DAVID V. TIEDEMAN 

11. lloumc^HMiv A. II, AND KEhkK II, t\ (!. Social cluM tunl mtnioliUnru, New York: 
Witfy« 1958. 

12. jACUon, D. N., AKU MtificK, S, Conlrnt and ttyle in pc^rtooality Mtc^ument. 
Piychol. Bull. SS, 2i3.2S2. 

13. Jamu. W. The true HArverd. In hlmoriu and Uudiu, New York;. Longment, 
Green, 19 M . Pp. 348-355. 

U. KKaT« C. A. Tk^pntMogyo/permmalconitrucis, Vol. I. New York: Norton, 19.SS. 

15. LnK« J. C, AND GlLCHftirr, A. A. Problems ol T. A. T* interpretation and the 
diignoaie of delinquenl trrnde. Brit, /. med, P»)choL^ 1958, 31, 51*59. 

16. PvecBU, K. The TAT end enlitocul behevior. J, ammit, Psychol, 1956, 20, 
449^. ^ „ 

17. Rkdl, F., anu WiNtMAN, D. Tht aggressive child. Clencoe, Illinoit; Frer Prest, 

1957. . . 

18. RmMiM, H. II., AND RAmm, D. H. The American teenager . Indiaiia|Milift, 
Indiana: B4»hlm'Mrrnll, 1957. 

10. RiUMAM, D. The lonely croud. New Haven: Yale Univ. Preia, 1950. 

20. RociSB, C. R. A proceM conception of paychotherapy. Amer, P$ychologi$t, 

1958, 13, 142-149. 

21. SANroRp, N., ed. Pertonality development during the college years. /. aoc. 
Im$i, 1956, 12. 171. , 

22. SifUiVAM, C, CranTx MARci'CiinK Q., AND CluifT, J. D. The development of 
inlerpcrsonal maturity: applications to delinquencv. Psychiatry, 1957, 20, 373*385. 

23. Tmj§, H. E., and Hollandeii, E. P. The Califiirnia F sc«le in psychological 
^ resewch: 1950-1955. Psythol fli*//., 1957, 54, 47 64. 



Discussion 
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The pi^ychologiHt has Imd what might be termed fair success in antici- 
pating a person*s later position on some scale Such as his over«aIf grade 
average in college. Even these moderate successes are often accomplished 
only after much investigation. Such investigations ordinarily requirr 
considerable trial and analysis of relationships existing among particular 
(lata of the past kfore deductions begin to square moderately well willi 
laler observalions. A half century of experience of this kind has caused 
Ihe modern psychologist to be higltly skeptical of propositions about 
relationships when such propo^^itions are not thoroughly che<:ked out 
beforehand. A peculiar fascination for empiricism has been the result. 
This fascination sometimes spawns ludicrous claims for the value of 
inductive empirical study in the absence of a specific criterion as in 
factor analysis studies. Dr. Loevinger s awareness of such ludicrous 
claims probably caused her to lash out, in good humor to be sure, at the 
method of factor analysis as she has done today. 
I could assume an air of rigliteousness and temporarily distract your 
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attention from Dr. Lor vjngrr's contribution by launching a defente of the 
method of factor analysis. Such action is inappropriate* though, because 
Dr« Loevinger gives every indication throughout her paper that she is a 
paychometrician par excellence. I prefer* therefore, to make several 
observations on limitations of the method of factor analysis. I shall 
attempt to do so by explication of the reasoning process in which 
Dr. Loevinger engaged. ! 

First, let us note explicitly the several aspects of the experimental 
method. In e^Hence, the experimental method consists of assembling a 
series of facttt in which it is observed that some antecedent circumstances 
are dissociated with some consequent circumstances* A theory is then 
evolved whi( h proposes that the generalized consequent circumstance 
has a functional dependence upon the generalized antecedent circum- 
stance. The tlieory pcfrmits deductions in the form of hypotheses that 
some unknown, but as ertainable, consequents will be of a certain form 
under certain previously specified conditions of the antecedents. 

These hypotheses then direct experiments in which the antecedents 
are created or found aad ^J^MBsociated consequents observed. If the 
observed consequents agree to an important degree with che deduced 
consequents, we have no presumptive reason to dismiss ||pi theory. 
Wlien the observed consequents fail to agree with the deduced ones to an 
important degree, However, we now have a new set of antecedent and 
consequent observations which must be Joined with our previous sets 
ond a theory must be invented that now produces order among these 
enlarged data. 

Now, let us turn our attention to limitations of the method of factor 
anaylsis. In terms of this paradigm of the experimental method it is 
quite apparent that the nietliod of factor analysis aims only at the 
rudiments of all that is needed, namely at the introduction of some 
simplification of either or both the antecedent or cotiM^quent responses. 
Further, the method of factor analysis is usually applied with little or 
no intent of later investigation in qpili and hence the simplifications 
resulting from a factor analysis bear no necessary relationship to those 
that may be needed in the construction of any theory. The purpose of a 
theory probably offers the l>est guide as to the relevance of one form of 
simplification or another. This is not known, however, until after a factor 
analysis is completed unless the factor analysis is itself a purposeful 
step in the formulation of the theory. 

We might note in this regard that the studies considered by Dr. 
Loevinger imply that she is more or less interested simultaneously in the 
processes of rducatifuu socialization and psychotherapy: By focusing her 
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atleiitiuti (ui tlit s«' pron ^^4•H ^imu|latir(ui8ly, and by extracting aipects 
of i-ertaiii ^^Uniii H, a.M hcII an |iy iiitroilucing crrlain obtfrvationi about 
children, Dr. Ltu^vinger LuiisirucH a reanonably convincing argument 
that we ought to con8ider suvh information in relation to an ability to 
i'oiueptualize one's Ht If. Thi scale nhe succeeds in drawing to our 
atlrntioa uttracis bnaii^e il orders, and hence gives meaning, to 
prcviouhly ilincrcle infornuttion. So far no factor analysis has attempted 
to deal Hith data for nurh divirsc! circumstances and over such an 
rxU'iidcd rangi* of a^v. In fai I, l)r. Loeviriger has amalgamated patterns 
of rrlali«inship at 1 1 liani^r that I, for one, cannot model mathematically. 

Vhvrv are asptM ts of Dr. Lnevinger's approach that fascinated me, 
howt vcr. Rcmrmhcr that Dr. Loevingcr is postulating an ability to 
ronccpiiialize oiti *m ^rlf, an ability generally characteriied by uncon- 
trolltMi impiiUr at itn Iuk point, f<i jmration from impulse but not from ego 
at the midpoint, and Hrparatiun from both at the highest point. The 
ability is prcsiimni t«) bi* a manifestation of ego development not related 
mniiotoiiically to sonu* other asppc is of ego development. It is interesting 
to notr tliat Dr. l orvingcr presents this scale as a Guttman-type scale, 
( 'HtMitially p(»slulating llir absence of the type-separation ^from ego but 
not !i-t>ni impulse. Hut what if this type does appear? I have a hunch that 
Dr. I iM s r kouKI dral with its appearance by designating a pors<»n 
of that iNpr ''unlii-aUliy**. Thus the app<*arance of the type would 
iM-ninir :t t aiNr fi>r ri*medial artti»n rather than a cause for rejection of 
tlir sralr. t'artor analysis, with its orientation to the patterns of re- 
-^ponscs thi insi lvrs, dot s not allow for sur h judgment. 

YUc inipiirtani e «)f lliis was brought home to me several years ago 
win II I ^as reflrrting npi>n tlir possibility of deriving Guttman-type 
^r.drs in 4iriMs of school acliie'Vrnirnt. It seemed to me that, in an area 
snrfi as arillinirlic, i fiildren were experted to master certain develop- 
uHMilal (asks in s('<pn*in i* and (hat, as a result, a Guttman-type scale, or 
at lca<*t a (^uiti iviNj M si air, wonld form around the developmental 
task^. Hut (hiMi I licf^an to nonder if childrens* responses in arithmetic 
would rt ally siab* in the (aittnuui srnse. Although I never endeavored 
to an^wt r (bis <|ui s(ion, I did ask myself, **What if childrens* responses 
don't \^u'\ tin* absenn* of a scaled response, on the part of Mome 

(hililrtn, pn snntpti\e evidi in e tliat the behavior of such children is 
tlilTiTi nt from tlif* intrnt of the teacher as scaled? Have I not gained 
thnrl y'/'' >i tuallv, I lu-lirvc I have, because I then possess two kinds 
of information: I. what (lie intvnded process of differentiation and 
intrgiatioii i<, and 2. in v^U'wU children the intentions are not a part 
of their brh isior pattern atui in which areas. This is diagnostic informa- 
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tion diiTerrnt from an andy»i» of the pieces without reference to the 
whole, information that usually results from the direct orientation to 
responses characteristic of factor analysis* 

In addition to the scale problem, a second fascinating aspect of I^r. 
Loevinger's reasoning is that, in attending to the ego, her thougltt 
nrrrssarily turnini to the perceptions one has of his person in relation t6 
HUih notions as tasks, ideas, and pi^ople. The evaluations t)ne holds 
of his person with regard to such relationships likely govern the way he 
orienu* liimself to situations, and hence his behavior. I doubt that an 
action by itself provides sufficient information for inferring the actor^s 
evaluation, however. The inference needs to be made from simultaneous 
consideration of the situation, the behavior in it, and the motivation for 
the behavior in it. The factor analyst has tended to limit his analyses to 
only one realm of activity at a time. Were motivation and behavior to 
he analysed simultaneously, according to the usual model of factor 
analysis, I would not know if the results would uncover the effect to 
Hhirh I am attempting to draw your attention. 

It seemA to me that dynamically oriented psychologists contend that an 
url devoid of its motivation cannot Ih5 correctly categorised with regard to 
itH meaning for the behavioral system of the one being studied. The factor 
anulytit diHHi not incorporate this judgment of meaning or function into 
\m factor computations. Rather, the initial data for the factor analysis 
^ould include a location of subjects with respect to some categories of 
behavior without regard for motivation, and/or some general categorisa* 
tioim of degrees nf motivation without much regard for situation and 
lH*havior in it. I »criou<ily doubt that an analysis of such data would 
rrvral factors close to the categories actually employed by an interpreter 
ronHidering situation, behavior, and motivation simultaneously. 

Finally, I want to note that Dr. Locvinger refers to a cognitive 
process, namely, . . a cognitive aspect of ego development." A 
cognitive proec»)i in uudrrntandable in terms of a succession of two 
Hubt«idiary proce!is|p, <lifTeren(iution and integration. Successive differen- 
tiation and integration create levels of response patterns according to the 
dynamically oriented p?»yclu»logi8t. These levels are apparent only from 
the persjicctive of the growth process; I cannot lee that they would be 
identified in a factor analysis of response patterns for a restricted 
range of age. 

At the outset I used precious moments to sketch a paradigm of the 
experimental method. I then attempted to indicate: I. that I saw no 
necessary n^anon why every categorization of information useful for 
psychological work need conform to responses as they exist; 2. that the 
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character of an act may not \ft understood suflic iently in terms of only 
the act itself; ancfs. that the resulu of differenliation and reintegration 
may obscure unification of relevant "-esponse because a higher level 
response may not appear sufficiently like its undifferentiated counterpart 
at a lower level. I have pursued both lines of reason because I want both 
to have my cake and lo eat it, too. 

In my judgment, Dr. L)evinger has offered no fundamental challenge 
of factor analysis; she has merely illustrated that it has limitations and 
that these limitations need to be acknowledged. 

In my judgment, also, Dr. Loevinger has been scientific in her 
approach. She bus dealt with a structure of data that factor analysis 
cannot analyze as hef^mind has done. The resulting structure she has 
given to an ability to concept naliae one's self is consistent with the facts 
she chose to consider. In additidfn, she has postulated relationships of 
this ability with other variables in ways that are subject to verification. 
Kach of theste endeavors is a part of the development of any science. 

It remains to be seen whether her postulates coincide with reality to 
an accept<>ble degree. In the meantime, her present argument convinces 
me sufficiently so that I will be inclined to consider responses of struc- 
tured personality tests in relation to her construct in thr future, namely 
lo consider the level of a response pattern in relation ta a subject^s age 
and to my judgment of bin difttanre from impulsivity and ego. 

I, for one, shall attend closely to Dr. Loevinger's investigation of this 
ability in the future, and would even attempt to investigate it myself 
if only she designate<» the locale more definitely than she has had time to 
do today. 



Goininenl by Jane Loevinger 

Dr. TiedemunV U!*r of ihe term 'Vrparalion from" one's impulses 
and ego suggests pathology rather than development and evokes connota* 
tions I strove to avoid. If there is an appreciable group ot people who 
achieve perspective with respect to thrir egos prior to achieving a 
minimal level of impuli^e control, the force of my argument is lost. 
I think of ego development as an organic growth process, a domain noti 
preempted by Guttman. Unlike in Guttman's scale model, any specific 
nunifestation is related to ego development only probabilistically; the! 
more specific, the lower the probability. Hence a rrurtal test of my 
hypothesis i^ not altogether simple. ^ I 




Measurement and Prediction of 
Teacher Effectiveness 

David G. Ryans, l>e(>artment of Educational Psychology, The 
University of Texas 



Proper consideration of cvci^ some of the more ohvioU8 problems asso- 
ciated with the definition, measurement, and prediction of teacher 
effectiveness would be, as you all well know; an exhausting undertaking. 
What I shall attempt is to sketchily review with you a few of the com- 
plexities facing researchers who try to study teaching competency; 
then go on and briefly summarise some high spots in research conducted 
by the Teacher Characteristics Study; and finally, suggest some tenta- 
tive conclusions about identifiable conditions and characteristics which 
may be associated with teacher effectiveness. 

Some Basic Issues 

The basic concern of research on teacher effectiveness is, of courAe« 
prediction. We seek to determine how and to what extent various data 
descriptive of teachers (e.g., verbal responses, overt acts, biographical 
ioformaiion, etc., all of which may be subsumed under teacher char* 
acteriaiics) are either 1. antecedents or 2. concomitants of some behavior 
agreed to be a component of some criterion of teaching competence. 

The extent to which such relationships can be uncovered depends, of 
cour e, not only on the real, or latent, relationships which may obtain, / 
hut also on 1. how unambiguously and operationally the agreed upon ' 
criterion can be defined, and how validly and reliably estimates of the ; 
criterion can be obtained, 2. how unambiguously the teacher char- 
acteristic under study can be identified and how validly and reliably it 
can be measured, and 3. what the purposes and hypotheses of the 
research are and how adequately it has been designed, taking into account 
sampling, control, and replication. 1 should like to deal briefly with these 
three areu of problems. 

Criterion Meamrement. Recently I attempted to outline different 
methods of obtaining criterion data relative to teacher effectiveness. 
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T\\r iiiajur ratrf»<>rirs iiM'linItMl; u) direct rfit'OHurerneiit of on-going 
ttvn hrr ht*haviMi (( .^ r tiinr sampling nvolving replicated systematic 
ol»rtrrvati<ui); !») indirect 'ri.*u.mrcmt*nt on preserved records of 

on filing tcai'lh'r In hjNMor (c^., tape recordings); c) indirect measure- 
ment, i>y non liMuiril ol»»'rY^»r^, loused on re<'all of teacher behavior and 
a^ser^HnnMit thrrt-of (r.g./ rntiugH l)y students, administrators, peers, 
rU\) \ d) ineuMtienuMil of a prnduel (student behavior) of teacher he- 
h.i\inr; and <•) ni(\nuirrnrnl roneon^ ' 'it;* (secondary criterion data) 
nl { \*\\rv'\\M\ III t»'a< Kri t'lTrrtiv i in 

riu'^^e <hlfrr«Mit a[>pnau'lu\H to i *asnrement vary in nature of ilitionale 
rniplnyed ti» ^nppiM t them, in n*Mal liity of the criterion data produced, 
and in the order iif nbtaine<l relationshi|)S between criterion estimates, 
thnn ditlrnMitlv diTiveil, and specified predirtors— -this last observation, of 
: MurM', merely Ix aiinj; lesliniony to the fact that most criteria arc very 
rniiijib x and ar\y one set <»f estimates is likely to he very incomplete 
u ilb Tt -^I'l • I \s) {\\r ivvrr;ill rritrrii»n. 

\pprt».^'brs to tlw nn a^in enieni of a criteiion of teacher effectiveness 
Ur.is iiivnivr ihi* rvaliKilion of eilHer 1. teacher b<'havior in process^ 
1, a inmiuvt ol Irai bcr behav.nr, i»r vonamitants of teacher behavior* 
Nb asnit mi nt of on-^Minj^ behavior of the teacher is the most direct 
i#|>prnarlr. nn jHureijfft ill of produ< lA and of concomitants ire less direct 
an I more sultjfM l U» llu* < n'rrls of eoiir<»Ui ding (^ondit'.ons. 

Cnnrninifanls (v\uS4j[^iM a sv\>m\ nmy be thought of as srcotidary 
rrilt'iinn ilal;i) n^^nally w iiol ar<-eplabb: for criterion measurement 
\sbrii tlirtM I iiK .l^u^eln^ III of brbavi<t. in process or tbc measurement of 
,i -ulabb* pr»Mbirls nt learbrr bi^iavior ran conveniently he used, irowever, 
in iMvfsti^atiiuis iiivolviii|^ ♦*\trn5ive sampling and where other ♦neasure- 
in«-t:t appro.\"hos arr ini[)rarlieab \\\v use of known correlates as sub- 
Hhtntfs for prorf^s or j^riMluri dala fre<)uently is defensible. 

Of il t^jasurtMiiriit approaclies eniploying observation and asseFsm«*nl 
Miilv time vimpUnf^ invohiiij^ rvpUcatrd systmuftic uhsenwiion hy trained 
nh\rnrr% pnuhn rs sufficiently uliable data to recommend iU^sc in 
fiMidaintrU.il romarib, aMIiough less well-confrolled variations (e.g., 
latioK^ bv ^IndrnN) may be t nipbiyeo when oaly coarse discrimination 
(i i:., ' lii^l" ami ' poort-it'' Irarliers with respect to s<mie criterion 
« iifiiMDnrnt) ir(p-.irrd. and wIumi the larger expected error is recognized 
and arri-plrd. \ arioOH as*<r-sinrnt lecbniipies have been dovelop; d, among 
vshii b ib^* nion^ r<'Iiable and promising appear to be 1. gr-^nbic scales 
with i^perationallv, or behaviorally, dclinetl poles and/or units, 2. obser- 
vation cliei k \m^. and 3. fnrced-eti(»lce scales. Tlie nhi^f shorfroniii^g of 
M}mfrv,i(i(»:i and a?*se<Hnie'it leefH!i(pie» ha? be^'M lack of reliability, a 
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i*lHMlcc)iniiig whi(?h rrsearcli has indicated can fairly wadily In? ovrrromc 
with rare to definition an(i to ncale development, and with adequate 
training (if the ulwrrvrr!i or judges. 

Produet meaMurements (eifttimateH of the liehavior or achievement of 
the pupiN of tearhern) have l)een widely acclaimed as deairable criterion 
data, but havr hc Mi iiifriH|uentIy used in tlie atudy of teacher effective* 
neHH, Artua!' y, the Heemiiig rolevanre and appropriatenew of the measure- 
ment of pupil U'liuviors and their products as indicptors of teacher 
performance n)ay he more appiireiit tlian real, for the producers of (or 
contrihuters to) "^pupil hehavit>r, or pupil'achievement, are numerous, 
and it is nnnt diilu ult tii designate and parcel out the contribution to a 
particular "product'' made by a specified aspect of the producing situa* 
tion, such as the tcarhrr. We also must note that the facets of the product 
criterion (vari'MH understandings, skills, and attitudes, etc. in various 
cirtitent fields and arras of personal behavior) ure similarly numerous, 
and each niiist be rapable of valid measurement and of at least |>artial 
isolation ft>r study. The comparability of estimates of various components 
or aspects of a product (pupil achievement, for example) also becomes a 
special problem wliVn student . behavior or achievement is employed as a 
criterion oUracUer efTeetivcncs>*. And when measurement of the product 
is aee<»mplisbe*l by obtiiining (*stimates of student change (i.e., pretest- 
posttesi data) the problem of variable potential gain (students who score 
high on the initial measurement being closer to their "ceilings'* than 
stmieiils \s\\^^ originally svore low are to theirs) is particularly plaguing 
to the researcher. However, if the rationale of the product (student 
perfornianee) criterion is accepted, and if the complex control problem 
presented by*A miulifdicity of producers and the multidimensionality of 
the critt»ri(»ii can be satisfactorily coped with, student change becomes ati 
intriguing apijiroaeh U> the measurement of teacher efTcetivcness. 

In dealing with any of the several approaches to measuring the 
criterion, the rirseareber must be thoroughly familiar with, and guard 
against, the various sources of erilericui measurement bias, particularly 
those wbieb have to do with a) incompleteness and b) contamination of 
the obtained data. ^ 

/Wn7r»r MniMiremnit. I shall pas> very «|uickly over the problem of 
obtaining es:inuitcs of the predictors. The diief technical pYoblems 
faced here are thijse familiar to educational research and measurement 
workers, namely validity <tnd reliability, e prediction of a criterion 
may be very seriously limited by the reliability of estimates of the pre» 
dictor empt^»Yr<f 4n a study. And unless the researcher has a pre^tty clear 
idea of the meaning of bis predictor eslimat'v and the conditions or traits 
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they actually rt^prifHf iit, iiitrrprrtatioii of predictor-criterion relation- 
nhipn may l>e pretty risky. 

It i» important to note that similarly named predictor measures (e.g., 
estimates of teacher empathy, ur leaders})ip, or understanding of chil- 
dren) used in different investigations do not necessarily refer to the same 
underlying characteristic of the teacher which is measured. Quite apart 
from sanipling errors, they do not necessarily yield similar relationships 
uith estimates of a specihed criterion dimennion. Discrepancies in 
lindingH reported in the literature someti^ies may be traced to this lack of 
iigreement in .op<*rtttional definition of the predictor, in addition to 
(•riterion inadequacies and lark of control of relevant variables, 

Research Objectit'en and Design, Still another set of conditions which 
roiitribute to variability in the nature and degree of association which 
may Ix* obtained l»etween hypothesised predictor measures and measures 
ol a criterion of teacher efTecliveness has to do with research objectives 
and the appn nch to the predictor crileiion relationship incorporated in 
the research oosign. Such queslioOA as the following should be (but 
frequently are n(»t) considered by the researcher. 

1. Does the investigation purpose to determine a) concomitant or 
b) antecedent-consequent relationships? 

2. Is pre "clion of the criterion of teacher effectiveness attempted 
from single bits t»f information (e«g., answers to a single question- 
naire, test, or inventory item) or from scores based on combinations 
of such bits of information forming sets of homogeneous items, or 
scales? (And, if the latter, does the combination of bits involve 
e(|ual or dilTerential weighting?) An extension of this question 
involves whether prediction of the criterion is determined from a 
single predictor alone or from a combination of predictor scores, 
Hcightcd perhaps in liglit of multiple regression weights. 

Is the derivation of predictors (original selection of items, or 
i i)nibinati(tns of items, ns predictors of the criterion) based upon 
experience Hilh a single sampli*, or has replication been employed 
involving mnltiple <<anqilcs of teachers? 

t. Ih prediction directed at a) additional random sanq)les of the same 
popnlatioii as tlie s wnples employed in deriving the predictors (e.g., 
ero^s \alidation) or b) samples of populations other than that 
Ireim \\\\\vh the predictors were derived either 1, enqiloying the 
sianie criterion measure (validity generalization) or 2. a different 
criterion measure (validity extension)? 

fi. Is prediction attempted i \r predictor data and criterion data which 
have l>rrti collected at approximately the snme time, or when the 
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obluining of crilt*rion liaia ha« been delayed and carried out with 
a considerable lime interval separating the coUcclii)n ol' the two 
•eta of et timates? 

6. Is prediction attempted when the predictor data are (»blained under 
"incentive" ronditioua (e.g., in ci iineriion with selection for 
employment) or undeP'non-iucentivr*' condilions (e.g., as in basic 
rcseareh)? 

7. U prediction attempted for helrt in! rrili iiiui diinrnsioiw singly 
(r.g., effective classroom discipline) or for a composite criterion , 
made up of a numlnr of hetrri>geneous c(»m(M>nenls pr dinu^nsions 
(e.g., overall teaching effectiveness)? 

8. (s predictiiHi of teacher rffrctiveness attempted on an actuarial^ 
or group, basis, or is the coiirrrn predict! mi for particular (iudivid* 
ual) teachers? 

Still other aspects of the prediction problem niiglit he noted, but these 
arc representative of s«>me of the major considerations involved iifi the 
overall design of studies ©f the predictor*criterion relationship. 

IVnchrr I'lTtrliveness .and tlic 
iVac hrr ( luiraclcrislics Study 

\V lirn, m ar llic br gioning of this discussinu, 1 referred to methods of 
tilitainiug criterion data relative to teacher eflectiveness, I avoided 
dcliuition of the term "teacher elTectiveness/' If I were pressed I might 
sav (hat 1 believe teaching is effective to the extent that the teacher acts 
in ways that are favorable to the development of basic skills, under* 
HtandingH, work habits, desirable attitudes, value judgments, and 
adetpiate personal ttdjustment of the pupiL But even such an operational- 
appearing definition really is very general and abstract and is not easily 
tran^latahle intt) terms relating specific teaelier behaviors. Kmbarrassing 
as it may be for prtd'essional c lucators to recognize, r<ilatively little 
pro^rivs has been made in rounding (»ut this definition with the details 
whicli are necessary for describing competent teaching or the char* 
arirrislics of eflective teachcia for a specific educational situatitm or 
etihiiral setting. Granted, most educators and most parents do have some 
idea of wlut constitutes effective teaching, The-r coneeptuolizalit»ns, 
hi»wever, usually are very vague and far n in<»ved from specific observable 
behaviors of teachers, frequently even sucli ha/y ideas e e highly in- 
divnluali/rd with very little agreement existing among <lifferent perH<Mi.4. 
(hie i** reminded of the old, familiar fal'le of the Mind vww who per- 
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reiveil ail rlr|iharit in witJely varying roannerf depending on the 
part of ihc flephi^nrn boiiy that each one touched. 

Reialivii) of Trmhrr Ffftctivene^. Diaagreement and ambiguity with 
rr^prt t to thr ilr^t ription of teacher eflectiveneaa are to be expected and 
< 4iuiot br (*;uir<*ly avoidtuilx'caufle compt'tent teachi. 3 undoubtedly ia a 
relative iiuiUrr. A p<*r7«oir» concept of a **good^* teacher depend^ 
firsi^ on iift uhuiatioii, \m paHt experience, and the value attitudes 
h(* \\an eonie In «irn'|>t, and, snvnd^ on the aspects of teaching which mily 
l)e foten^ist in \m rormiileration at a given time. Pupil F, therefore, may 
(lilTer nideiy from pupil C in his concept of the essential attributes of an 
efTective tearhcr. If pupil F is bright, academically minded, well adjusted 
un<i in<i(*peniient, he niuy valuer most the teacher ^ho is serious, rigor- 
ounly ara(l(*rni(*; ami perhaps relatively impersonal. If pupil G, on the 
other haoii, in more sensitive ami requires considerable succorance, he 
may foul the teat In r just described not at all to his liking and indeed 
literally "iinpo.s-iihlt*.'' In the itiind of pupil G, the better teacher may 
very hcII be one who U nomewhat le^H exacting from an academic 
8taii(lpoiiit, but who is (lianirleristically sympathetic, understanding, 
and the like. 

Answers to the qu(*stioo, ''What is an effective teacher tike?** also 
may vary to a ilt*^ree with tbe particular kind of a teacher one ''hooses 
to consider. It dues not >eem unreasonable to hypothesize that, even if it 
were [lossibh* ti) agree upon a generalized definition of effective teaching 
whic h woulil W aret ptal>lo to a number of different cultures, and if our 
thinking might be ohjei tifie^l )io t!ie point where effective teaching could 
Im* divirnUil oifu luetual basis, ''good** teachers of different grades or 
(hlTerent Hubj, « t matters still rhight vrry considerably in personal and 
social < harai^teristirs and in varioo.« domains of classroonr behavior. 

The ii»nrept ot eoi tpetent teaming must therefore be considered to 
be relative to at least two majcr sets of conditions: 1. the social or 
cultural group in which the terchcr operates, involving social values 
whi<'h rrecpiriitly difTcr from person to person, community to community, 
culture to culture, and time to time, and 2. the grade level and subject 
matter taught. It is not suprising, then, to note the difnculties that have 
confrontwl those seeking to establish criteria of teacher effectiveness, 
the dearth of testable hypotheses produced in such research as has been 
undertaken, and a general lack of understanding of the problem of the 
characteristi< s of effective teachers. One very important reason why 
effective or inefTective teachers cannot be described with any assurance 
is the wide variation in tasks performed by the teachers and in value 
concepts of wliat ctuistitutes desirable teaching objectives. 
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But addition to these conniderationi, and important in its own right 
M a deterrent io the study of teacher effectiveness, is the fact that there is 
M lack of\any cleiir knowledge of the patterns of behaviors that typify 
iiidividual\ who hvt rmployed as teachers. It seems probable that, 
without losing sight of the importance of developing means for recog* 
niiing '^good^ teachers, attention of the researcher might first more 
properly and profitably be directed at the identification and estimation 
of some of the major patterns of personal and social characteristics of, 
tf achers. This reprrnents the point of departure for research conducted 
Uy the Teacher Characteristics Study. 

In the Teai her Characteristics Study, considerations of the effective* 
nrs^, or value, of particular teacht;r behaviors were to a large extent 
disregarded. Instead, attention was focused on the study of possible 
teaclior l>ehavior dimensions, such dittiensions being hypothesisedito 
represent generalized trait coutinua. From this point of view teacher 
behavior variables are assumed to consist of clusters of relatively 
homogeneous (positively intercorrelated) b«|iaviors»-.such component 
behaviors being uf the nature of !^mplf predicates, capable of operational 
d(*finition. 

Implied in this approach i^ the af^tmimption that a teacher may be 
(Irscribed in term^ uf a po»itioh on a particular l)ehavior dimension, 
^ui'h desrription being ensentially factual and relating to observable 
manifestations of overt behavior or else to responses known to be 
I'orrrlatetl with some behavior pattern to a degree that may permit 
indirect estimation of the behavior. 

The Teacher CharacterisSics Study, The Teacher Characteristics Study 
w^s sponsored by the American Council on Education and generously 
i^upported by Tl^ Grant Foundation. During the six years of the Study 
spproximately 100 separate research projects were carried out and over 
6000 teachers in 1700 schools and about 450 school systems participated 
in various phases of the research. Some of the basic studies involved 
rxtensive cla*sruom observation (by trained observers) of teachers, with 
the purpose of discovering eigniticant patterns of teacher behavior. 
Other activities of the project had to do with the development of in- 
struments (paper ami pencil tests and inventories) for the identification 
of individuals characterized by different levels of specified patterns of 
a) classroom l»ehavior, b) attitudes and educational viewpoints, c) verbal 
intelligence, and d) emotional stability. Still other investigations were 
concerned with the comparison of defined groups of teachers (e.g., 
elementary teachers and secondsry teachf^rs, married and unmarried 
teachers, etc.), from the standpoint of their (observable characteristics. 
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Bisically, the Tfti hf r CliaracteriBticB Study had ihnt major purposes: 
1. to analyze and deHcril>r (Mtttfrna of teacher clasiroom behayior and 
the manifestation* of certsin value systems and cognitive and emotional 
traits of teachers; 2. to isolate and combine into scales significant 
correlates (provided by responses to self-re{)ort inventories foncerned 
with the teacher*s preferrncest t iperiences* self appraisals, judgment* 
and the like) of some major dimensions of teacher behavior; and 3. to 
compare American teachers (in termii of the teac her characteristics 
described by the Study) when they hod \>ee\\ classified according to a 
number of conditions. 

« Pursuance of these objrj^ves involved development of techniques for 
the reliahlf rt^'^f^nnu iit of classroom Ixliavior, determination (largely 
through .uc. ^r iL::ilyHim) of some major patterns of teacher behavior^ 
dcvrlu| Mt \ inttruiiients made up of materials hypothetically related 
to teach H u Hi lu ,a behavior dimensions and other personal and social 
characterhv 4 ' '>f tearherii, the empirical derivation of scoring keys for 
^urh instrui cnis in light of response-criterion correlations, and finally 
comparison of dtTined group? of tcachorH. 

Pnlterm ofClassfoom Behavior As a result of the direct observation and 
assessment of teacher classroom behavior and subsequent statistical 
analyses of the medsuremcnt data, several interdependent patterns of 
teacher l>ehavior "^cre suggested. Three in particular appeared to stand 
(»ut in separate factor analyses of elementary and secondary teachers: 

T.(^.S. Pattern Xo--understanding, friendly v$, aloof, egocentric re* 

stricted teacher behavior 
T.C.S. Pattern Yo--re9ponsible, businesslike, S} matic M. evading, 

unplanned, slipshod teacher behavior 
T.d.S. Pattern /o^^timulating, imaginative, surgent im. dull, routine 

teacher behavior 

Pattern scores X„, Yo» and derived {win observers* estimrtes of 
teacher b«*haviors in the classroom, appeared to possess sufficient 
reliability to permit comparisons of teacher groups with respect to these 
patterns and, aNo, to justify their use for criterion purposes in attempt* 
ing to identify inventory responses which migbt be used to predict 
teacber classroom bi^havior. 

Among elementar/ school teachers, patterns Xo^ Yo and Zo were 
higbly intert orrelated and each also seemed to be highly correlated with 
pupil l>ehavior in the teachers* classes. Among secondary school 
teachers the intercorrelations of the patterns were less high, than 
between patterns X„ (friendly) and Yo (organized) being of a very low 
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order. The teacher cliMroom behavior patternt and pupil behavior were 
much lots highly correlated among secondary teachers as compared 
with elrmentary teachers. 

Elementary and secondary teachers, as major groups, differed hardly 
at all with respect to mean assessments on patterns Xo, and Zo. 
However, grade 5-6 women teachers, represented by a relatively small 
sample, were assessed somewhat higher on the several classroom behariar 
patterns (particularly on Yo) than teachers of other elementary grades. 
Among secondary school groups;, social studies teachers received the 
highest meaa assessment on pattern (friendly l>ehavior) and women 
mathematics teachers (with woben social studico teachers not far 
behind) on pattern Yo (bu<tii esslike behavior). Teachers over 55 years 
of age received distinctly less high mean assessments on pattern 
(friendly), sad slso slightly lower with regard to pattern Zo (stimulating), 
than younger teacher groups. Among elementary teachers the mean 
aHttessroents on the classroom behavior patterns Xo, Yo, and Zo were 
nlightly but insij^nificantly higher for married as comprcd with single 
teachers. Among secondary mathematics-science teachers, single teachers 
rrtH'ived higher mean assessments than did those who were married. 
With rt\npect to English-scHial studies teachers, single teachers were 
assessed higher than married teachers on pattern Yo, but slightly lower 
on patterns X„ and Zq. In general, differences between teacher groups 
compared on the observed clasnroom behavior patterns Xg, Yo, and Zo 
were not pronounced. However, it is of interest to note that scores on the 
IVacher (Ihararterintics Schedule (to be descril>ed shortly), based on 
ki ys (X,.„, Yc .n and Z,.o) derived to predict these classroom behavior 
patterns, frequently distinguished different teacher groups more sharply 
and with greater assurance than did the Xo, Yg, and Zo observation data. 

l^aiiernn of JVi/wes, (vrlkd Ability^ and Emotional Stability. Inevitably 
the Teacher (lliaracteristicH Study sought other evidences of teacher 
behovior in addition to those provided by assessments of overt classroom 
behavior. To extend the understand.ing of conative and cognitive aspects 
of teacher behavior, and to permit tlie mote complete investigation of 
relationships between teacher characteristics and specified conditions 
of teaching, the study undertook a number of researches directed at 
analysers of teacher's attitudes^ their educational viewpoints, their 
verbal intelligence, and their emotional adjustment, and attempted to 
develop direct inquiry type instruments for estimaling frotn a teacher^s 
respotises his status relative to such behavior domains. 

In one set of studies a number of opinionnaires relating to teachers* 
attitudes toward groups of persons contacted in the school were developed 
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and the crganizalioii of tracher attitudes waa studied through factor 
analysis. In keeping with the results of the factor analyses the study 
centered its attention chiefly on the attitudes of teachers toward pupils, 
their attitudes toward administrators, and their attitudes toward fellow 
tt*achers and non-udtninistrative personnel. 

The educational vifwpDints of teachers with respect to curricular 
oiganization and scope, pupil parliripation and class planning, academic 
a* liirveqientr^ standards, etc. also were investigated (separately for ele- 
nitMitury and settondary teachers) through the employment of direct 
iiH|uiry type* items and factor analysis of the intercorrelations among 
r(*MpoiiHrs, Tlio patterns of viewpoints which emerged were not clear«cut 
and then* ««(*rmed to lir some justification for considering teachers* 
(*(lucational l»clicfs from the standpoint of a single continuum, over 
•iniplilitMl pi^rliaps l^y its designation as a *'traditional-permissive** 
(Innrnsion. 

To (»l)tain c^tiniatc.H of the verlial understanding of teachers, vocabulary 
and vci hal analogy itcm<t were constructed, experimentally administered, 
and the rc^sponses aii;ilyzed, the procedure culminating in the selection 
of a ^maW nunilK*r of highly dii^criminating items comprising a ^Verbal 
aliilily'' >^cal(\ In a sitnilur way materials were prepared and analyzed to 
ohtaiii ittniH f(»r providing (*HtiniaU*s of the emotional stability of teachfrs. 
And (o aid in llic dt ttu tion of ^tendency to make a good impression** 
wlicri dt*alirig with rtsponsen to direct question type materials, a set of 
ilrnis intended to measure probable validity*of*response of teachers also 
\^vis n<iscmlil(*d. 

Various Htutlit*s anti comparisons of the attitudes, educational view- 
|i(>irUs, v(*rbal uiulrr^tanding, and emotional adjustment of teachers were 
uiidt rtaken ir> ilit* coorst* of thc^ development of such measuring devices 
ii^ tliost^ noted altMVt*. Some of these results were extremely interesting, 
but I shall iiul atlenipt to go into them here. 1 shall move on to a 
drst ripti(»n of onr efVorts to obtain indirect estimates of teacher class- 
ronni beliaviors and other characteristics from correlated inventory 

An Inirnton for Indim t Estimation. In llie interest of providing more 
readily ohtaioaMi^ estimates of teacher classroom behaviors, and also 
( stimates ol teacher attitudesv viewpoints, verbal ability, and emotional 
stability wliit >i migtit be less susceptible to the response set of giving 
soeially acerptable responses* efforts of the Teacher Characteristics 
Slutly were ditet ted at the derivation of correlates scoring keys applicable 
to tlie items of the Teacher Tharacteristics Schedule. The Teacher 
( liiinirteristirH Schedule was an omnibus self-report type Inventory 
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hated upim some 25 originally separate instruments. In iU final form, 
it consinted of 300 multiple rhoice and check list type items relating to 
personal preferences, self-judgments, frequently engaged in activities, 
biographical data, and the like. 

Employing as criteria a) observers' assessments of teacher classroom 
behaviors X„, Y„, and Z« and b) scores on the direct response scales 
relative to teacher attitudes, viewpoints, verbal intelligence, and emo- 
tional stability, hundrinls of response analyses were carried out (thanks 
to SWAC, our first high speed computer at UCLA). Response-criterion 
rorrelations wrre obtained for each response to each item of the Teacher 
(lharacterisiics Schedule under a variety of conditions. Correlates 
scoring keys, employing responses associated with the criterion behaviors 
as sign^ or nymptonid of behavior, thus were derived for a large number 
(»f teacher groups. The most generally applicable sets of scoring keys 
(and those most frequently used in other phases of the study's research) 
were the all-elementary teacher keys, the all-secondary teacher keys, 
and the combined elementary-secondary teacher keys. 

Reliability data for the correlates scoring keys and various kinds of 
validity data, relating particularly to the friendly (X), business-like (Y), 
utid stimulating (Z) keys were obtained. Generally speaking, the reliability 
eoefTu ients fell lietween ,7 and .8 and the validity coefficients were of 
varying magnitude depending upon the kind of validity investigated, the 
particular behavior entimaled, and the teacher group from which the 
key was derived and to which it might reasonably be applied. Concurrent 
validity coefficients for correlates scores on classroom behavior patterns 
X, Y, and Z typically were between .2 and .4; predictive validity co- 
efficients were positive, hot generally low, seldom exceeding .2. Inter- 
correlations among scores resulting from application of the several 
correlates scoring key» estimating classroom behaviors, attitudes, educa- 
tional vi-^wpoints, verbal intelligence and emotional stability, and 
( orrelations between Schedule scores and observers* assessments, 
indicated 1. substantial relationships among the correlates data and 
2. prediction of observecl classroom behaviors principally by the scales 
specifically developed for that purpose (X^.o, Y«,„ and Zoo). 

''liigK' and ''Low"' Tenchcn Compared. I shall not deal here with the 
numerous comparisons of teachers which were made in light of the 
Teacher Characteristics Schedule data collected. But I do want to mention 
a study we c(»nd acted which was concerned with identifying teachers who 
fell into one of three groups: one group comprised of teachers each of 
whom had received observer assessments one standard deviation or more 
above the mean on each of the three classroom behavior patterns Xoi 
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Yo« tnd Zo\ another madr up of teac hers who were all within two-tenthi 
of a standard deviation on either side of the mean on the three different 
classroom behavior patterns; and a third group made up of teachers all 
of whom received observers* assessments one standard deviation or 
more below the mean on each of the three classroom behavior patterns. 
After haying identified these teachers w<* attemptiHl to determine some of 
the distinguishing characteristics, in terms of Teacher Characteristics 
Schedule rrnponi^es, of the different groups. Here, I suppose, we were 
approaching the problem of over-all teacher effectiveness. We were 
attempting to discover responses of generally highly assessed teachers 
which distinguished them from generally lowly assessed teachers. I shall 
summarise some of the more notable characterL<»tics, for combined 
elen^entary and secondary teachers, which distinguished the high group 
from the low and the low group from the high. There was a general 
tendency for ''high*' teachem to: be extremely generous in appraisals of 
the behavior and motives of other persons; possess strong interest in 
reading and literary affairs; be intereited in mo^«ic, painting, and the arts 
in general; partiei|>ate in social groups; enjoy pupil relationships; 
prefer non*direetive elas.sro(»m procedures; manifest superior verbal 
intelligenee; and he ulu^ve average in emotional adjustment. Turning to 
the other side of the coin, "low** tearhers tended generally to: be re- 
Htrirtive and rritical in their appraisals of other persons; prefer activities 
which did not involve elose [H*rsonal contacts; express less favorable 
opinions of pupils; manifest less high verbal intelligence; show less 
satisl'artory emotional adjustment; and represent older age groups. 

<)l>viously, the ileseription I have been able to give of the Teacher 
(lharaeteristies Study very sketi hy. I have noi been able to get down 
to some of the really very interesting findings such as those related to 
eomparison. of teaeher groups and interrelurionships among teacher 
behaviors. I >vilK however, he able to incorporate some of our findings 
in the concluding section which f«»llows, 

Sdiiio IVobublc (!orrolato8 of Teaclirr KlTi^^tivoiioss 

It is indeed presumptuous and dangerous to speak out boldly UiUiut 
condkions and teacher characteristics associated with teacher < ffcctive* 
ness. However, based upon the findings of various researches conducted 
by the Teacher Characteristics Study and an accumulation of investiga- 
tion' which have «p})eared in the literature over a period t»f years, certain 
threads of fact do seem <liscernil)le. Hut the* conclusions and inferences 
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an* Hlill, at brsl, lenlalivr— they aro more in the nature of hy^olheBt^ 
for which tum< support has been found in our American midtwentit th 
century culture. And we also must recognize that changing educational 
values in the future reasonably may lead to changes in the patterning of 
tracher behaviors and to further revision of our unJcrstanding of 
predictors of teacher cffcctivrncHH. 

The following generalizations r^'garding the r( lationship between 
teacher characterintict*, as predictors, and teacher effectivenesn, as a 
< riterion abstracted from various criterion measurer* reported in the 
literature, appear to Ik* in order. 

Characteristics and conditions of tin* teacher which are likely to he 
positively ctnrelntvd or a.'tsociated with leinlu»r eflectivenefts in the 
abstract include: I* measured intellectual abilities, particularly verbal 
intelligence; achievement in college courses; general cultural and specific 
subject matter knowledge; professional infnrmation (knowledge of 
education and teaching); practit^e teaching marks; emotional adjustment; 
attitudes favorable Vo students or pupils; genenmity in appraisals of the 
behaviors and motives of oilier personri terest in reading and literary 
matters; interest in music and painting; participation in social and com* 
mnnity aflairs, early experiences in caring for children and in teaching 
(e.g., reading to children, taking class for teacher), history of teaching in 
family, size of school and size of community in whicli presently teaching, 
and cultural level of community in which teaching. 2. Kxtensiveness of 
gtMieral and or professional education, enrollment in particular pro- 
b^ssional courses, and personal appearance appear to bear very little 
relation to the abstracted criterion of general teacher efTectiveness, 
3. tJementary teachers and secondary teachers, as groups, do not seem 
to differ greatly when an over-all view of teacher cfTectiveness is taken. 
However, elementary teachers do seem to show superiority when 
selected aspects of criterion behavior having to do with warmth, per* 
missiveness, atul favorable attitudes toward children are considered. 
Secondary teachers are superior from the standpoint of verbal under- 
standing. Within the elementary school. Grade 5-6 teachers tend toward 
snt>eriority on several criterion dimensions; within the secondary school, 
tlnglinh and social studies teachers show A similar tendency. 4. Age of the 
teacher and amount of teaching experience seem to manifest an over*all 
negative relationship with teaching effectiveness, although there is 
evidence of curvilinearity, increase in effectiveness appearing to be 
positively correlated with age and experience during early years of 
teaching careers. S. Sex differences in (iver-ali teacher effectiveness Ao< 
\v i appear to be pronounced, but the ela^room performance of women 
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tfachers ttfrins to bf innre organized and businesslike than that of men, 
and men teachers seem to be very distinctly more emotionally Itable. 
6. For teacherii of ail grades and subjects; considered together differences 
in effectiveness bet%ire<*n single and married teachers are small. However, 
within the riemenury school the evidence appears to favor married 
teachers. At the secondary level results are somewhat mixed, with un- 
married teachers as a group appearing to be superior with respect to 
such criteria as busiiiess-iike classroom behavior, permissive viewpoints, 
and verbal understanding, but with married teachers showing superior 
emotional adjustment. 

Certain chaiarteri<(tirs, then, do seem to be associated with certain 
dimensions of teacher behavior and teacher effectiveness, although the 
extent of obtained relationships frequently has not been high. It is 
important here to recall that 1. relationships and differences which 
have been noted are in terms of averages for groups of teachers, and 2. 
any obtained relationship is limited by', and may be expected to vary with, 
conditions such as those noted in an early part of the paper. The useful- 
ness of research findings pertaining to the prediction of teacher effective- 
ness will be greatest when the results are considered in an actuarial 
context, rather than in attempting highly accurate prediction for 
given individuals, and when variations in relationship found among 
different classifications of teachers, and with use of different approaches 
to the predictor criterion relationship, are taken into account. 

Appendix: Predictability of Teacher Effectiveness 

The notes which follow have to do with general considerations relating to 
conditions which prolmbly should be taken into account both in the 
design and the interpretation of research on teacher effectiveness. 
Some of these arc derived from rational analysis of the problems involved, 
but many also have subntantia! support from empirical data. 

1. The predictability of teacher effectiveness undoubtedly is affected 
by the malti-dimrnmmUity of the criterion. There is accumulating evidence 
that prediction can be accomplished with better than chance results for 
!^peritird dimcnHioMH or components of the criterion. On the other hand, 
the prediction of over all teacher effectiveness is possible only to the 
extent that some general agreement can be reached regarding the 
dimeni><ions comprising ''over-all effectiveness^^ (involving, of course, 
acccptarNM of a common *<ei of educatifmni values) and how they should 
be combined in form u comptiHitc. Tcnchers effective with regard to one 
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upect of the critrrion may not he effective when judged by other 
criterion dimensions. 

2. The prediclability of teacher efTectiveneBs varies depending on the 
degrtf of control it is possible to exert in dealing with the multiplicity of 
predictors and the multidimensionality of the criterion. 

3. The predictability of the criterion varies with (he kind of measure 
employed in obtaining the criterion data* 

4. The predictability of the criterion varies with the adequacy 
(reliability and validity) of mfasurts of a) tho criterion and b) the pre- 
dictor variables. 

5. The predictability of the criterion is so limited by conditions associ- 
ated with measurement pf the criterion, measurement of predictors, and 
practical conditions, that relationships representing common variance of 
perhaps one-hfth or one-fourth of the total variance probably approach 
the maximum to be expected except in chance instances, 

6. The predictability of a dimension of the criterion of teacher 
eflectivenesH from a specified predictor probably varies depending upon 
the cultural milieu which provides the setting for an investigation, 
particularly the values and objectives prominent in the teacher training 
curriculum at the time the teachers^studied were in college. 

7. Predictability of the criterion varies directly with the degree of 
similarity between the sample with respect Xu which predictors arc 
derived, and tho sample to which the predictors arc applied in attempting 
to determine predictur^criterion rclationshipH. 

8. Predictability of a criterion dimension varies with the particular 
teacher population (e.g., Grade 1-2 women teachers, men science teachers, 
etc.), or student population, studied. Effective teaching methods may 
differ frt»m one grade level to another and from course to course. 

9. Predictability of the criterion varies inversely with the time interval 
separating the obtaining of predictor mcasnrements and criterion meas* 
ureme Us. 

10. Predictability of the eriterioii probably varies depending uppun the 
ai^Huoiatioo u( incentive or non^incentivr conditions with the obtaining of pre- 
dictor data. 

11. The regr( ?^hioii of predirtor meaHoiements on criterion ^^eusure- 
ments frequently in curtHlinear (e.g., positive (*orrelation between amount 
of teaching experience and certain criterion measures of effectiveness of 
secondary school teachers during firnt five years or sen followed by 
leveling nfTand decline in ei iterion estiinutc^H with extensive experience), 

12. Pre<lirli(»n of teacher efTeciiveheHs must be considered largely in 
the actuariftl ^en<ie: individual pn^dic^tion, an generally is the case in 
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atlt rni fo prnli^ l humati Ix hii -or^ is rniu h mure limited and is 
•ii'i nm|ilishrd with a IrMir dej^rec* i)t' ronfidriii*t*, 



llvKKV H. Cii.HKKT, HiKinl ()f KxaniiiuTs, New York City Board 
of KJiiralioii ' 



TIk' nl a iiimijssuni in fnlhivving Dr. Kyatis' papi'r is unenviable, in 
\\\ w\ ifMperts, biit >i dislinc'. privilrge in iifliers. First let u» examine tlie 
urjrnvialile aH}>eef-«. i)r. ./mis has given as a spltiididly condensed 
\' isi(»n t»f th<» «'rnijirrs,-<r({ results of a .v.M'ies of Heparate rei^eareh 
^luJirs ranging fri»ni elaliDntf^ to cumplex— studiei, extending over a 
pri ind 4»f yr.ir^. invr , thjuisanils of teacluTfl and at least 1700 Hchool** 
in ir>n ?*<'hMol systems. |)nes the diseussatil prnet^ed to address himself 
lo pritMrniH of nirtl)iMliili»gy, terhniijnes, ii:**ereni'es fr(»m the data and 
I 111' hki*y I think y(»u nui^t share my ftM'Iing t)iat I should not, even if I 
wnr i\\ur iH. riu' srupe v\ too vast to he trcuieil thuH. 

I ani remindcil of a sifjn tfiat grt*(*ls ine regularly in my favorite coffee 
^hop. It reads "I'se \nnr head---it's the lil U* tl ings that c(»unt'' and that 
I- prfM'jsrly my esliniute of sell in relation to the magnitude of the studies 
nmlrr roiiHideraliuii. I have a deep respect for Kyans and his colleagues, 
and the^nhjtMi they liave hern studying, and, I fiope, a proper apprecia- 
tion itf httle ohi nn* in that big i»ld context. 

1 h«\r then*! ore chosrn to discu.ss siunc general implications (»f Ryans' 
Ntuthes, mi\ her'* I ft»el di*eply privileged for the opportunity- 

Snrely 'here is no need to dwell overlong on the social signihcance of 
the stud, en. All tif us are aware (»f the sliortiige of teachers. Wc are 
av\are nt the ithrinking supply t»f future teachers in our colleges, and of 
t)nr inert rtsirtg pupil pii^iulalion. It spells trouble now and deeper trouble 
iihrad for all of ns '^\u) see education as the country ^a pressing concern 
and basie approaeli to the deveiopmiMit of a sound nation and world. 

riierefure, f^ur ihosc of u^ who are directly involved in the selection of 
learfnM^ He loojv with great interest to all res earch that can be of help 
III {I*, We in tin* New York (lity school system are in big bt^siness in 
ff\n her '-ein linn. M e examine about 30,000 applicjition« annually and 
need rephn enp nts al all levels numberitig in the th(»U8ands each year, 
W»' Imm aililn HHfil nin>»'Kes io the nroblem of determining scopes of 
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exumiimtinii<9, constructing tenttt, setting pass marks in advance* some- 
what unt*asily aware of the tremendous complexity of devising prediction 
instriHiUMits for iinestablishcd and undefined criteria, 

Ryans' resu give us much cause for concern. He points up the 
difliculties in determining teecher efTectiveness, particularly in attempt- 
tng to compreAs the many dimensions into a single over*all categorization. 
Yet we go right on making our tests designed to predict overall teaching 
success. 

AnotluT point. Only recently, the Superintendent of Schools in this 
city has f^vwv on record favoring an advance in nalary for teachers of 
''HviptTior merit/' There are many school systems throughout the country 
tliat have sucli a policy in cITect. lloweveff if there is one concluaion'we* 
can draw from the studies under discussion it is that the plain facts 
indicate wc eainnit Meh*ct such '"superior merit teachers*^ with ^ny degree 
of conlidcncc. The idea of superior merit is intriguing. However, the 
resullH of objective iiivcHtigation should give pause to school adminis- 
triitor!<, entirely apart from tlie host of other arguments that can be 
lirouf^lit to bear against the adoption of such a proposal. I refer, of course, 
to llic negative niorab** factors, the' inevitable rivalries among teachers, 
the invitation t(* currying favor and the like. 

Hnt alt is not l>lack: One notes with a sigh of relief some of thd 
comparisons of "higli" and "low" teachers that are reported. I call to 
your attention that ''higir* teachers ii^ .e been found to be superior in 
verbal irilclligcnrc, interest in reading and ahove'average in emotional 
adjustment in contrast witli "low" teachers who are inferior in these 
dirnrnsions. I have l)een deliberately selective in citing these char- 
acteris^tics. Our selection procedures are loaded with thcje variables, 
riiis is not the time to discuss the validity or roliability of the instruments 
we ui^e. It is of im^jorlance to be reassured, for whatever it is worth, 
tliat our selection procedures are designed in what appears to be some 
desirab'te dintcnsions. 

(Vviainly rbc conclusion— the very obvious conclusion— we must 
male Is tfi u the studies s|u»uld invigoraUr us with the determination to 
cifntiriiK* the research that has been started. It should make us feel 
v.'ry humble about oer .too.rnlrenched, too*f stablished procedures and 
iill us M ith tlic coovic;.ion tliat we must not be floored by the complexity 
of the problem. We )iave to leorn the d^mensioM jf teacher effectiveness 
and how to predict them. Tlii^ may mean an*enhc^ly new approach to 
selection, but at least wc must proceed in the direc tions that the research 
InuU, and not, as we have been doing too frequently, ]:M*occed in the 
directions of lest eonstruction in whi/h wc are most competent or in just 
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plain intrrvicwiiig, which our rgC3 will not permit us to question. 

Tlie current »tagc i» most opportune for extennion of this research. 
It i» prohahly true that most people who Complete teacher training can 
find johs us teaiiiers. Tlit*y may not all he placed in the precise locale they 
desire. But the shortage 18 such that jobs can he found. Note the tlifference 
from the situatinn in the I930's, for example. In that decade, the job of 
, tlie teacher was highly sought TIktc !vrts wholesale attrition among 
applicants. Any research then on'predictiiMi of teacher effectiveness was 
severely limited hecause of the large turn^away. 

The situation, I repeat, is radically different today and in the near 
foreseeahle future. Hence it would be most urgent to institute a crash 
program of Hignihcant dimensions to extend and refine Ryans* work. 
WitneHH the current struggles betwe.n llie liberal arts and teachers' 
college advocates. Note llie presumpiaous assertions by laymen regarding 
education and the preparation of teachers. 

The trouble, as vwr. is that those who proceed cautiously and are 
nWare of the C(»niplexity of the problem are least likely , to male their 
voices heard. 

I end wilfi llie ni»te t|iat we havlp a public responsibility to make known 
our need to untlertake a vast program of research, designed to provide 
fundamental guides to teacher training, teacher selection and inevitably 
to iu«sei vice training and supervision of teachers. We arc indebted to 
Kyans and lii^^ cidlengues, but we shall let them down, and let ourselves 
down, if we do not insist on the logical continualion of the work. 
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Luncheon Address 

Some Observations 
on. Soviet Kducation 

by Henry Ciuuncey, Pre»idenl, Educational Testing Service 



A tiumber of ypurn ago, when I was talking with Crneral liersliey in liin 
office, he suddenly »tupped, turned to me and said, "You are probai)ly 
like mt — just liavt; one siirech. Sumelimea I begin at the beginning ami 
go to ihe rod; sometimes I begin at the end and go to the beginning; 
Homelimes I begin in the middle and go both ways. That confuses them, 
but tlu y thijl^ it's profound/' 

I used that story to introduce General tiershey at an Invitational 
('onferenee that some of you may remember, and 1 use it again today. 
For certainly, with regard to RuHsia, I have but one speech, and I am 
afraid that I may seem to enter it in the mi<ldle and go in several 
directions. I hope, however, that it will not be confusing and I hope 
further that you will not think it profound, since it represents merely 
the olmorvations of one visitor to the Soviet Union. I happened to have 
the goiKl fin tone to be a meml)er of the Prst American educational team 
to visit Russia under tlie i'ultural Exchange Agreement. The team went 
c»ver under the ao?*pices of the Office of Exiucation and was led by 
Ijiwrence Derthick, the United States ('ommissioner of Education. 
It was our ahsj;;nment to look into eb^meiitary a» d secondary education 
anil al^o teacher training. 

We spent a month in the Soviet Union and traveled fairly widely. 
After a week in Mimiow, we went to Kazan, the capital of the Tartar 
Republic; Sverdlovsk, in Siberia; Alma Ata in Khatakstan, near the 
Mongoluin Ijorder; Tashkent, in U^^bekistan, across from Afghanistan; 
to Sochi on the Black Sea; then to Minsk in Belorussia, and Leningrad, 
then back to Moscow for linal visits with various people in the Ministry 
of Eilucation. 

Our h(»^ts^ mollified the itinerary and plans they had made for us 
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to suit our wishcK nml gave um opportunity, within practical iimitii 
to »rr whaU'ver we wuiitril to see. While I feel sure lhal wc did not 
visit some of the worst schools in the Soviet Union, we saw a broad 
range of institutions. Since in each ci^.y the team split up into smaller 
groups, we were ahlr to visit a large number of institutiona--well 
over a hundrcMi. 

I think I might hegin by indicating the three things that struck me 
uioMt about eduintion in the Soviet Union. First, and most important 
of all, is the tremendous commitment of fhe people of the country to 
education. Secondly, the flexibility of the country in educational matters, 
as in other ways* And, thirdly, tho great progress that has been made 
ill edu ration during the last forty years. 

Till* foremost impre»»ion tliat our whole team had was the strong 
ronimitment (»f the Soviet J>eople to education, their conviction that 
thiA v^ tremtMid(»usly important, and their desire to do everything to 
improve education and strengthen it. 

Now, the reason for this is that they believe education is the founda- 
tion of national piiv^cr. They believe that if ihey are going to be strong, 
srientiru ally, militarily, economically, they must be an educated country, 
educated \\\ niaiiy difTevent ways, and that this is the basis on which they 
will gr(»w from power to power. 
They are a country with a tremendous power drive, and their aim, 
they express it on bulletin boards all over the Soviet Union, is 
to "veaeh and surpass America.*' This is their goal, and education is 
the foundation stone. 

A SI rood quality, flexibility, that struck me forcibly also came as' a 
8urpri*4t\ One thinks of Russia as a dictatorship, a monnltthic enterprise 
that in moving ahead relentlessly in one direction, a direction that will 
lifter prove to be wrong. Hut the fact of the matter is that the Russians 
are tremendously flexible and adaptable. They move ahead, but if they 
find eooflitions re({uire a difl^cTcnt taek, they make it. 

When I ,talkeil to Henry Shapiro, the UP correspondent who has been 
thrre llf) yeavj», comment was ll»at Russia is the most flexible country 
in the world. IVcjple jui^t don*t understand or recognize this, but the 
Soviet V\\UM\ in enntinoalty changing and adapting, moving^ ahead, 
moving asifle, as cMMiditiiuis make necessary. This is true in education 
an well as in many other ways. 

The third observation that I want to mention particularly is the great 
pn^l^res^i *he Rus^^ians have made in their education over the last 40 
yi ars. One could not but be impressed by this, because they Ulk about^ 
it all the time; nevertheless, they have facU and figures to back it up. 
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They suited with « large number of obittacies to overcome: « tre- 
mendouB touiHry, sparsely populated, with t very small educational 
system that was in ten led particularly for the nobility, for the elite, 
They wrre 70 percent illiterate in 1917. They didn't have an industrial 
economy to back up any progress that they might plan. 

And then in the midst of their development during the last 40 years, 
they had what they call "the Great Patriotic Wav," and the wholr 
western half of their country was overrun and devastated. Despite this, 
Ihey have continued steadily to make progress in the field of education. 

Let me give you an example from one Republic, becanse I think 
specifics usually make the situation a little clearer. In the Republic of 
Uibekistan. down near Afghanistan, the population was' 98 percent 
illiterate in 1917. There were 160 schools with 17,300 pupils. Today 
illiteracy has Iwen virtually wiped out in Uzbekistan. There arc 5,800 
schools with 1,300,000 students. In 1917 there were no institutions 
of higher education; today there are 34. 

Education in the Soviet Union has gone through a number of phases 
since the 1917 Revolution. It is useful in understanding Russian 
education tinlay to recapitulate briefly the steps that have been taken 
since the Communists come into power. Before the Revolution, the 
Russians had a European type of cducolionol program. It was strictly 
academic, very rigorous, and intended for 'the iritellertuul elite, not 
for people of the country as a whole. After the Revolution, the Com- 
munists threw the whole system out and decided to have educotion for 
everybwly along "progressive" lines, then in vogue, in sorte countries. 

But somehow they carried it a little too far, and in the e^rly 30's 
they began to l)e disillusioned by progressive education. They found 
that it was not producing the kind of trained individuals for scientific 
work or for other kinds of leadership that they needed, So, with their 
customary facility for making an about-face, they discarded progressive 
education completely— ond with t, incidentally, all use of objective 
tests wliich the progressives had Introduced. They have never reinstated 
oltjeclivc testing. 

In 1931 they adopted a plan for a rigorous academic program, but 
not, as in pre Revolulion days, just for the elite. Now it was to be 
the course of Htudy which everybody would follow. When I say "every- 
body," I mean on the order of 99 pi r cent, exriuding approximately 
one per cent of the population that may be mentally defective for 
physiologi. al reasons. It is the Russians' belief that all others, handled 
properly and given the proper training, can he educated, even in such a 
rigorous academic program. 
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When the RuMitnt adopted thii new program in 1934, univeraal 
t ducation extended only through thr fourth grade. Ry 1950, they had 
uuiveraal and compulsory education throughout the Soviet Union 
through the seventh grade. Since the seventh grade in Ruaeia is about 
comparable to the ninth grade in this country, this meant, essentially! 
education througjK what we call junior high school. They had planned, 
before Khiushcl^ev^s recent announcement, to provide universal^uca* 
lion through the tenth grade in all republics of the Soviet Union byl960, 

Disregarding the recent annouiuTmi iit for the moment, let us con* 
(«ider the nature of the Russians* educational program today. It Has« * 
I would say, two major objectives. First, there is the goal of providing 
general education of the academic type for all i.udents through the 
tenth grade, to supply the Soviet Union with a vast reservoir of people 
who are capable of furthrr education along any line that may be necessary 
at a kter time 

The seroiid ol jn tive '\s to provide vocational training, training for 
i«pmtic job!4. ThiH may be training for a Mcnii-^killed job in a factory, 
wbcre there a fairly short iiourse, or it may be training of a research 
|»by?<iriMl, whirb involves university and postgraduate work and a very 
lottg eoiirse. The Russians huve a tremendous number of different kinds 
or edueationnl programt^ and educational institutions, geared to training 
people for variou<« oectipation.^ and usually murh more specifically 
oriented lowartl a partieular job or eareer than would be true in' this 
eountry. 

Let mr deseribe, very quiekly, tbis Soviet school system. It starts 
Hilh tbt^iur^ery nrhools, where ebildrcn from six months to three years 
'go wbib' tbeir nintberft work. Next eome kindergartens for children 
Irorn ibier to nix years of age. Neitlu r the nursery sehi«ols nor the 
kindergarten* are universal tbroiighont ibe Soviet. Union yet, but they 
are expambng very rapidly. 

After ibe kindrrgnrten lomes tbe Ten-Year School, for children 
from seven to 17. In about half of tbe Soviet Union at present these 
HehooU only go tbrough the seventh grade, after which students must 
eontinue their edueation by g<»ing to an evening school, a Technicum, 
A labor reserve m IiooK or by correspondein our.«*e^. 

Ill the remaining 1 n-Year Sehoojj*, wbirh do go through the tenth 
grade, graduating HtudentH go on to higber education or to some kind 
of vocatioilal eduration. riu re are t^o kind^ of institutions of higher 
eilneatioo. tbe imiversitieH and tbe ni<ttitutes. Tbe universities offer 
wnrk in sueb at ademic subjert» as pby^irH, rbenii^try, history, linguistics, 
i»nd HO on. The instituten art^ the profesj^iona) training institutions, except 
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ill Ihe case of Ian, nliirh lor antnc n annn happniH to be tflHiuiatrd wilh 
ihf uiiifertitifs. 

Studentt who do not go on to highrr education may go to X\\r ^()ecial* 
iifd vocational tchoulf for a short courts, or to a Tfchiiicum for a 2'^- 
year course of technical training. It it interesting to notf that the RuHmann 
have far more of these than we do in this country. We have, it has bern 
estimated, only about 75 technical institutes in the United States, and 
there are something like 4,000 in the Soviet Union, with T)ver 2,000,000 
students enrolled. Obviously, the Technirum is a major part of their 
eiiucational systrm. As I iiidiratrii before, not all Htudrntn Ksit until 
the end of tlu» tenth grade before entt ring tlie Trcbnii uni. Some enter 
after the seventh gradt , but they take an academic program comparable 
to that offered in the eighth to tenth graden ol the Ten-Year School 
along with their technical training. 

There are some extracurrii ular activities in Russia that are extremely 
important in the education of their abler students, and it would give an 
incomplete picture if ! didn^t mention them. Soviet students have clubs 
which they call *'circU^*/' Sometimes these are associated with^hc 
schools, sometimes with the Pioneer Friaces, which are youth centers 
Homewhat comparable to. our YMCA, Boy Sc its, and other activities 
all rolled into massive proportions. • 

At the Pioneer Palaces there are "circles'* for such activitiea'as art, 
inuHic or shop work. There are aUo **circles** for academic subjects 
Mucb as phyHics, mathematics, and chemistry. The leaders of these 
groups are usually associate professors or instructors in the univer- 
sitif .s. They encouragt* students interested in these special fields, and 
they try to spot the most able students and bring them along even 
faster. Often they encourage the able student to try out foi' the Olym- 
piads, which are academic tournaments by subject matter fields that 
bigin at the local levcPand progress through the district leVel to tin* 
Hepublic level, to the all Soviet Union level. These very highly couh 
petitive Olympiads provide a way of encouragiii|^ and developing out- 
standing students that is lacking in the regular Russian school program, 
where everybody takes the same er idemic courses. 

Now I should like to describe in more detail the program of the 
Ten-Year School itself, since this is really the heart of the Soviet 
educational system. As^I have indicated, it is an academic program and 
a pretty rigorous one. Perhaps I can be a little more specific if I take 
the la^t three grades of the Ten-Year School and average the number 
of tinicn per week, a student attends classes lit a particular subject, to 
give you a picture of a typiral year in the laj^t three yi ars of this school. 
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riu» rtluilf III wciulil have, on the average, »ix classes a week in math- 
eniutic}^, lour in physiiH, three in chemistry, one in hiology, five in 
literature, four in history (principally Soviet history), three in fo&eign 
languages (preceded by three years of study in the same foreign language, 
making a total of six years), two in geography, one in technical drawing, 
and two in physical education. 

This, clearly, is a rather stiff academic program, yet the Russians 
expect 99 per cent of llieir students to take it, profit from it, and complete 
a. The question i»: to what extent is this true? To what extent arc they 
abli» to get 99 per cent of their students through a program of thia 
nature? 

I was unable to get complete statistics to provide a definite answer to 
thin qncHtiiui, but trying to make the best estimate I could on the basis 
of tlic cvidmcc available, I figured that somewhere between 50 and 80 
♦ per cent of Russian «^tudents actually get through this Ten-Year School 
program. In addition, others take a somewhat similar but perhaps slightly 
watered-down version of it in Technicums or labor reserve schools or by 
rorrespondciuf i our?eft. This is something that is hard for us to under- 
stand, because we have generally thought that only 15, 20, or 25 per cent 
of our students could take such a program. 

How flo the Russians carry their students through these strictly 
iirmleinif subjects? A iiunil>er of factors might be related to the effective- 
ness of their jiro^ram, and I shall review some of them quickly, some 
ill more detail. 

Certainly the scbixd luiildings in Russia are no asset to the educational 
system. Their buildings, as everybody has reported, arc drab, ordinary 
buildings witli nothing fancy or modern about them. The same is true 
of school equipment such as chairs and desks, which arc equally old- 
fashionru. But when it comes to laboratory equipment, movie projectors 
and screens, and slide projectors, the story is different. These arc widely 
availalJe and. on the wliole, although I am not a terribly good judge, the 
equipment for laboratory experiments and the machine shop equipment 
seemed to nir to be very good, aifd a real asset to the educational system. 

Perhaps the strongest asset, however, and a key factor in the success 
of the Soviet educational pr<»gram, is the Soviet teacher. Teaching is a 
very iiltnictivr profession in Russia. There are four or five applicants 
for every positi(»n in a teacher*s college. The tcacher*s college, which 
ij^caHcd the Pedagogical Institute, has a five-year program with thorough 
training in both the subject matter the individual is to leach and in 
metliodsand principles ol teaching as well as in educational psychology. 

Througliout their rureefs, teachers are expected to spend one summer 
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(lit of three ia further study and training. They get one day off out of 
the Bix-day school week, and are supposed to devote this time to their* 
prufessioiiul devplopment. They arc also supposed to do some outside 
reading in their field and report on it from time to time* In addition, 
there are teachers* clubs ii: many cities where teachers may go to 
consult specialists about pedagogical probIems.\ 

The normal teat hing load, somewhat lighter titan it is in this country, 
is 18 hours a week for the full-time teacher. Some do teach more than 
that, but they are paid extra for it. TeAchers, however, do a considerable 
amount of special tutoring beyond their classroom work, because they 
are held responsible for the success or failure of their students and 
therefore put a lot of efTort into individual work with students who are 
nol doing wril. ^\ 

Textbooks and teaching aids are another factor in theVfTectiveness 
of the academic program in Russian schools. These are dovel^iped in an 
institution for which we have no parallel iii this country— the^cadcmy 
of Pedagogical Sciences. The Academy of Pedagogical SciencesX which 
Works through eight research institutes and has a total of 550 res^rch 
Workers, pluys a very important part in the educational program in the 
Soviet Union. It has accrss to the leading scholars and scientists through* 
out the Soviet Union, and can call on them to cooperate with the Academy 
Hiaff on all sorts of educational problems. 

One of the eight iiHtitutcs in the Academy of Pedagogical Sciences 
is, the Institute of Mdhods. This is the Institute that coordinates the 
preparation and development of new texllio^iks and other teaching 
materials. When a tcxtbfiok needs revision, it is sent out to many 
teachers and to the leading scholars in that particular field, who are 
asked to comment on ti e textbook. Then the Institute asks a scholar or ^ 
scientist who is usually one of the top people in his Held to work with 
the Institute staff on the writing of the new textbook. 

After they have worked on the prohlem for awhile, they put out a 
100-pape summary of tlie nt?w book^s contents, and this is also distributed 
to teachers and leading scholars for their reactions. After studying these, 
they begin to write the textbook, working on 50*page sections at a time, 
tlach 50-page section then goes out to a substantial number of experi* 
mental schools, for trial and evaluation of how well the book gets 
specific ideas across to the students. Again, revisions are made, and 
finally the textbook if^ completed. 

Then it has to go before an independent commission appointed by the 
Ministry of Education to determine whrther it is better than any other 
text that any otiier individual in the Soviet Union may have written in 
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the same tirld, and whether, in fact, it should be adopted for general use. 

These books, if they are in the field of science and nuthematics, tend 
tt) be used throughout the Soviet I'nion. If they are in history and 
literature, they aren't necessarily adopted in all the republics^ because 
some of the republics want to rmphuHiie their own literature and history, 
and they have the privilege of Hutmtitutiiig their own textbooks. 

Sound films and other visual aid^ are also prepared by the Institute 
of Methods, and are drsignrd to supplement the textbooks in getting 
the most difTu ult ideas and concepts across to the students. The Insti* 
tute has developed something like 80 films in physics, 30 in chemistry, 
and about 100 in biology. These films are an integral part of the respec* 
live courseH and are »hown in all Hchool<«. In addition, a school will 
have all kinds of ( hartH. I have never seen so many charta. You can go 
into any classroom and there are charts on thr walls depicting important 
points the teachers want to get acnm?*, with several hundred more charts 
stored in nearby eh»?*ets. There are also tliree dimensional models to 
help visualize difficult concepts, and monographn available fur students 
hIio want to read further in a subject. 

ThTs multitude of teaching aids is all a result of the work of the Insti* 
tute of Methoils. It seemst to me that in preparing these materials the 
Russians have taken a lot m(»re trouble than we have to make sure that 
Mchfio! textbooks and teaching h'uU cmlxuly the best and latest thinking 
of the le'^ders in the field at the university level along with the best 
tem^liing methods tliat those on the front line of teaching and in the 
Institute of Methods can devise to communicate these ideas to students. 

( >ne more reason why the Russians are able to carry a large proportion 
of young people through an academic program— and I think they get 
too many through, as I will explain later— is the motivation of the stu* 
dents. Their motivation is extremely high. Success in education is very 
important to the Soviet student. He can pretty well predict what his 
income and prestige will tte 20 years later by the grades he is getting in 
school. Salaries in the Soviet Union arc usually fixed in accordance with 
how much education is required for the job. 

(^dlege professors, incidentally, ire among the highest paid people in 
the Soviet Union. The only others who are paid more are the top party 
officials and government leaders. Teachers in elementary and secondary 
schools are also well paid. There is some diflfe rence of opinion as to how 
well, but on repeat^ occasions we were told that their salaries compared 
with the salaries of engineers and doctors. In any case, they mutt be 
rea9onably good, or there would not be so many applicants for every 
opening in the Pedagogical Institutes* 
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SiJ muih tor the fmlors influencing tlif fffectivcnrM of the Soviet 
eduMational i»y«lrm. The question now in: Whut hasi been the rcBull of ihis 
I'orced draught fdoration, thirt ma^i^ivf rffort to give the entirr ,;op- 
ulatiun m aradmir program? 

What has happtMicd is thai in rerent yearn the Ruttsians have been 
training many more students in a college preparatory program than can 
poHisibly he admitted to a university or an institute. Although they have 
been graduating about U5(H),(KM) students each year from the Tcn-Ycar 
SchooU, tbe institutions of higher education can accommodate only 
about 250,000 lull-time sfudentn aud 200,0(X) parl^ime students. Thus 
only about oui' in thret graduates of the Ten-Year Schools is going to be 
able to goiulf> higher rdurati(»n. Yet all of ihcm have Iwen inspired with 
this as a gotl, ami thcv have j*la\tHl through work that for many of them 
was terribly difficult in order to attain that goal. 

Obviously, llir result is grave disappointment among thosq who cannot 
get into a university or an institute. They have to go to work, where they 
are needed* l)Ut they are disgruntled as workers, and they are not 
really prepared to do the kind of work that is required of them on farms 
or in industry. 

When this began to become evident a number of years ago, the 
Russians introduced into the Ten- Year School program, on an experi- 
mental basi**, what they called polyteibnical instruction. Throughout 
the elementary and secondary grades, this instruction is given in addition 
Xo the regular academic program. Course » in handicrafts, woodworking 
and metal working arc given in the early grades. In the later grades, there 
is machine ^hc»p, work wilh electrical machines, and p<»riods of actual 
work in industry or agrirullure, with students going perhaps two days 
a week for a certain period of time to work in a factory or on a farm. This 
polytechnical itistrut tion was supposed to introduce students to actual 
fartorv or farm work* or enable those who would go on lo higher 
ed»( ation to know how the other half lives. 

Hut the basil' problem still remained: many students were unhappy 
l)erause their hopes (ov go'*ng on to higlr r education were not fulfilled; 
and the economy desperately needed more workers. So in April 1958, 
the Russians tried another idea. Khrushchev announced that thence* 
forth all students finishing the Ten-Year Srhocds should work for two 
years before going to the university or institute. This would get more 
people into the labor market and would give all students, even those 
who would eventually enter higher education, a chance to know what 
factory or farm W(>rk i-^ like. 

It was never really inten<led that this policy would be put into effect 
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10() |>t*r cent, because the Rustiianti knew that they had to let their 
niathrmaticians and scirntUts continue their studies without interrupt 
tii)n. The Tact of the matter i» that 50 per cent of the students admitted 
to the UnivtTHily of Moscow and the University of Leningrad last year 
came directly from school. But the announced policy made the students 
who had to go to work ^ .*el that this was policy, and that they were doing 
something that was both good for them and good for the country. 

Since last April, however, the Rust^ians have begun to realize that 
neither the polylechnical program nor requiring ;^tudents to work two 
years before going on to higher education was going to solve the grave 
problem thai lay ahead of them. The Soviet Union is up against a 
fliluation that is completely different from the one we face. We arc 
coping with u big bulge in the birth rate that began during World War II, 
Biid is just now filling our high schools and colleges with more students 
than ever before. 

Kussia, on the other hand, stiffer<!d a decline in the birth rate during 
the war years, and therefore has fewer rather than more young people 
coming through \[$ educational system and entering the economy at the 
present time. The figures on this are q\iite dramatic. A year ago, there 
were 6,2.50,(X)0 17-yrar-olds in the So. hi Union. Next year, just two 
years later, there will be only 3,250,000 Soviet youth in the 17-year-old 
age brncket. Of the 6,250 000 17-year-olds a year ago, 1,500,000 were 
regtilar students in the tenth grade of the Ten-Year Schools, leaving 
4,750,000 available for work in factories and on farms. But next year, 
wh<Mi there will be only 3,250,000 17-year-oMs and 1,500,000 of these 
will be ill sehool, tlierc will remai*i only 1,750,000 available as workers 
--3,0()0,0(X) les^ than they had a year ago. 

In view of the rapid rate at which the Soviet economy is expf ??.:' , > 
it is clear that the Russians must have more workers avoilable in;r.t Ji- 
ately and caim:>t afford to allow so many students to r;> ' .iuc with their 
education at this particular time. This constitutes /'A^ve pcpblemt 
and, 1 anspect, is oiu, of the frctors behind Khrushchev's laiest announce^ 
nient. A Yuxk mo^e than r month ago, in September 19^1, Khrushchev 
issued fiome new proposals which, he said, were set forth by the 
Praesidium of tliC Ci n'.ral Committee of the Communist Parly. He also 
suggested that these proposals be discussed up and down the land, so 
that reactions could be considered before final plans were adopted. 

The basic idea of the proposals boiled down to this pronouncement: 
*'AI1 boys and girls without exception will ;o to work after the eighth 
grade, and do their studying on a shift basis or in the evening by 
correspondence.** It' this were actually put into effect, the shortage of 
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young people available for work would obvioubly be Boniewhat relieved^ 
because there would be a couple of extra age groups quickly poured into 
the labor market. 

But there are two interesting points worth noting in this latest 
Khrushchev announcement. First, he contradicted within the same 
announcement the statement **without e: 'caption/* and said that the 
very ullc students in the arts and muh \ in mathematics and the 
sciences, would continue to study full time at special schools* 

Second, he state*d that tht)se who go to work after th;; eighth grade 
and later apply for admission to a university would be admitted on the 
basis not only of examinations, but ai;^o of recommendations from the 
trade union and the Young Communist League. 

Khruslichcv noted that 70 per cent of those going on to universities 
now had come from families of intellectual workers and office workers, 
and only 30 per cent from the vast population of industrial workers 
and farmers. He declared that these figures must indicate something 
wrong in the admissions system of the universities, and that the situation 
should be corrected. What he did not say, but probably meant, was that 
lu* and other Communi?«t leaders are concerned about the possible 
dcvehipment of a class society, of an intellectual elite resulting from the 
fart tliat intellectuals are having children who become intellectuals, 
and that, sooner or later, these generations of intellectuals might 
threaten the Communist Party leadership. 

It HtM ins likely, at any raU\ tliat two new hurdles will be introduced for 
a large proportioi of Russian students applying for admission to institu- 
tions of Ingher education: they will have to get the approval of the trade 
union, which means they must hav? worked hard and been good workers, 
liked by their fellows; and they will have to win approval from the 
Young Conmiunist League, which must feel that they are cither good 
potential Party members or at least tlie kind (»f people iii whom the 
Parly can have confidcrire. 

Now, 1 doubt that these developments inilicate any h'ss devotion on 
the part of the Russian people, or the Soviet leaders, to education. 
What they do indicate is that Russia has to contend with some grave 
pvoblenis, purtitudarly the shortage of workecs, and that it nuki.c make 
some adjustnienVt in its educational system, at least for awhile. These 
development^^ may also indicate that the Russians* ov(*renthusiasm for 
general educalitni (»r an academic type for everybody, regardless of 
ability, is going to be tempered. 

Members of this conference will be interested, I think, in two topics 
I have not yet t(Miched cm: guidance and examinathns. 
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if^uUwM'v, a- ii is kiKiHii in tlii> ciiuntry, in noiiexiHiciit in Russia, 
(IihI in Curt Wnu is nnt tnnl<*rstiMi(l. Tliry dn not iisi' ivbin and tliry 
<Ih not hu.r ^iMiiiiih*»* ♦MMjn-irhn s. I hrrt* is Irss iirrrssily for rilluT wince 
fvt'fv «*tuiJri.l Uikrn f\a( llv \\\v suHU' rigidly uradrinic program. Tliey 
<'o , ln>Hr\rr, oulsidr ol' \Ur rrf^ular .scliool [)roj^runi, a Hyntem whirh 
uii^ht I'C rallt'd ''f(uidan*'r hy i'X|iosurr/' This is [\\v trrmrndous variety 
id MpporUinily lor "xira l un iridar arlivilirs fillfrrd hy ilxr "circlt^a/' 
or rlul»s, ill \\\r I'ionci'r Palarrs. Sludi'iils liavr anipio opportunity to try 
out poH^ililr nilrrr-^is l»y juiniiii^ oni' or anotluT td* lln»so rlubi*. Since 
tlir rluhs start at lln- first f^radc Icvi'i and conlinur t.> tenth grade« there 
'iir opportunities for cacli studoiit to lind an artivily of interest to him 
and in ^^liii h \\v ran do well. This, so far as I rouki lind, was tlie unly 
iiirihod that tho idissian^ have for guithiig students into special lields, 
Kxainiiiations in Kus:'a are greatly ilincrcnt from ihos'/ used in the 
I nilcil Statc'i. and seem wvy strange ti> those of us familiar with the 
kindn nl' ^'xaininalion^ that are used in this country. Olijeetive te^ts 
rannot Im* loiirid in Kii^sia, except in a few psychological luhoratories. 
1'Im y Here iMe.i in the early year^ of the (^.oinmunist regime, as I men- 
tinned earlier, hnt were thrown out in the early thirties ahnig with the 
progre*«sive.^ who had used thetn. The principal kind of test used in this 
earlier prru»d was the I(/ Since, hy its very namf\ an Inti^lligenc* Test 
iniplii*^ tin* gM iil iinporlain c of heredity as oppiise*l to training, the ii^ 
Je-l conlradicicd Tarty dortrinc. In the St»vu*J Urion, it is the firm 
coiwiriiun ef tiny **[U'^^**'*^ ' hackground and training, 

p.ulicniarls tin* Litter, are far nmn* inipt»rtaiil than heredity. They 
leeogni/e that individual ditferences d(» i^xist, hul claim that all itaii- 
\idiiaU ran he luniiglit to a remarkahie level (»f achievement hv prop<'r 
tf.n ling. 

In irsis not iMil\ contradicted *heacce[itcd doctrine hul, even vorse, 
operated to ^eh'ct the Hnldrcii of inttdlectnais rather than tin ciuldren 
nl i!ir .avnred Svoiker-./* When this C(»ntiiine> t«i happen M«flcr a 
g( peratitMi o| the ('oiiiinncist regime, it meant either that the Com- 
inuin^t^ had not provided the kind of society and education f^r tin* 
"wnrkers" that they luni set out to pnividi^ or that their thci»ry was 
wri»ng ai: i inlu rited jhility was, in fact, id' fiiaji>r import'Uice. NeitluT 
( fOiehiHion wii^ aci i'ptahle and all tests w« re haiiished. ^ 

11 tin* Hussiaris d«»ri't use ohjective t<*sls, wl* it kind of "xamiiiationH 
do they use and what do they use them U*\'f Tests are not used for 
idciitihcatitm. guidance or placeiiieiit of individnals as they are in 
this ronnli'v. The main use of examinations in Hunsia i^ io audit achieve- 
nniit. Priin ifial rclianrc is placed (»ri (»rul examinations of a rather 
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'*! thrs^ .Ajins st riii^ liinitni ti» uHsiiriii)^ that the' eHHeniial t*lt'' 

Kxaiuiiu»tiiiiH prrjiart'd \\\v Ministry of ^duration uHed to Le 
f!i\rM at tlie Viid ol' rucli grade, hu : |(radually thin ''^)rograni han iM^n 
itMliirtMl until at thr prrsrMt time exaniitiatioM^ are {fiveti ojdy at the 
'.'lul rf tlir Mevnitli grade dnd at the end of the tenth grade, There are 
aUo exaiiiiiiaf'oiiti for adiuiH?4io.i to the univernitieH atid the inntitutes. 
They Li e t|iiite similar ^o, tlioogl) more demanding than, the exaininaiioui 
^iv(*n it th(* end of the tenth grade. 

At the end of tlie He\.^ntli grade, all students take a writV'n examina- 
tini and an oral examination in |{uHHian ana a written examination iiiv 
al^^'hru. At tlie end 6( the tenth grade, tfiere are written examinations 
4M KusMan literature ami rompoiiition, aiuj oral examination in mathe^ 
malie.H. |diy>ii's, rlirmisf^y and Soviet hintory, Thr unu*^iw\l feature of tlie 
Hurssj.in ( \i:rniuationH can |)erha|i> best Im eAplained by deKcrihing one 
Ol d anil one writtfMi ex<ifnii;atiou. For the tentli grade oral exan^iatimi 
Ml solid geonu try, for example, a elass of 30 in divided into tw6»ecti6n» 
ill IT) cai h. All 15 students go into a elassrtMr.ii where the teachei;^of tlie 
t*lar*?».ihf» direi tor of the si'|um>K one or two other teacher?*, and some- 
times repn^M nt'itives of the edurationni aiithoritieH of the eity sit 
as an t*xamining hoard. 

On the table, at whu h the hoard sits, are 20 to 25 ''tiekrls/' each 
of whieh (ontaiiis three questions. IVo of theip are rather standard 
proofs i»r [irohlems and tlu' tliitd a problem Unit, v>liile not new in type, 
i^ perhaps a little diflerent from the pndflnns thai have b^en assigiu*d 
(hiring the year. All tln>e 'piestioiiH are <'oiueriud with $ partieular 
lopir, and eai h vf the 20 U) 25 ''tiekels** rtJVerH a different topi<'. 
Stuilents tluMrfore have to be sure that they have covered ^11 the 
topics, siuct^ tliey imm never tell in advanct* which questions they may 
be called upon to answer, hirtuuately, for thc/n, the Ministry of Kduca- 
hoii publisln> a panipldc^ several mtmths bt^fore the examinations in 
which the tn|>ics to In* ctivcrcd an' listetl, and in some instances the first 
two questions are spccilically stated. Only the third |)roblem or question 
remains iu doubt. 

At the bcgiiuiing nf the examination four or (ivt* students select 
their "lii k» tH.** 'i'hen thev return lo their desks and work out the 
answers. \\ In u ihe lirsi student is prf par<Ml, he goes to rtie blackboani 
and writes tuit bis answers on the bh^ kbctard. He then explains his 
answers to the (^xanuneis and responds lo any (piesticms they may have, 
The examiners may ask queslions about any aspect of the (MUirse, but 
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I gallu r lluil ihis privilrgp in oxrrrisr(t lo any f;roat rxlnil. Mean- 
Hln|t\ ol rournr, slmlriiln lw(», ihriT, ami fniir liiivr had coiiHiclvraMe 
liiuf In prrparr llu fr answrrn, lull at llir sai^t' lime ihry liave liniJ a 
j^immI tiral t»t tli^lrai Uon: SUkUmiIs 13, ai>^ K) Imvr lu sil llirough a 
li»ng miM iMMf^ *»f wailiiij^ until {\\v\t Iuiiuh nuuv \\\h * 

U is fviilriil frt)ni llirir rxaiiunatii)n ny^\vn\ lhal llir Hussiann'fncirs 
allf'^iUiHi o\\ thv iiuislrry uF impurlartl U^\nv^ Id a much giralrr exlriil 
than wv i\o in thin rnunlry. On iho olhn' Iruul, lIuTc ii^fur.lcss incrnlive 
til slnily niurr than thr minimum v -^jMitials. The rxamination» arc 
primarily a nn»livalitig ifrvire. Thi-y proviilc a goal towaril which the 
stu<h nt-stnvi vV. In thr srhnuls wc V'sitrd, it was rurr tu find thai any 
studi-nt had failed his rxaminatinriH the iMvvions year. Teachers may 
[O't vent ^tmlent** frmn taking the examinatinnH if they have had a low 
averapMlnrinf the year, hnt as fdr us we et)nld ascertain nnly one ur two 
sUiilrnts in a class <d':U) wuuM he prevented ftom taking the examinatioriH. 

On '.viiiten examinations all students arc presented with tl;- »amc 
ipn ^liuns. Kaeh i^tndenl must answer only one out of three qncftiiona. 
The It. pie- ?'Mid to l>e very grneral. Stuilents write an essay of not more 
than alMtm eight Uiir [took pages. Surprisingly enough, they liave six 
hi»ui^ in \shirh tt» (h> this. The exaniination in Russian literature, which 
I nl.srrvtil, iM'gan at 0 (/clock in the morning and ccnitinucd until 
:{ p. It was < nsti>nuny, and pt^rhap^ even tnamlatory for^^u*|i student 
tn \Mile a lii-t. draft ami then, having worked it <»vrr very carefuify, 
<u»py it iM-fure tlw* cnti oi thr examination. Our' own expcruMiee with 
twsiy <'\.iminatit>ns would lead us to que?^tii»n the' time limits set for 
11, ' cxaniinalion, anil the reliahility of an' examinalion involving only 
a single question. However, the Uupsians* ohjectivc jieems merely to 
de termine v.hetl^pj students have> met a crrttiin tiiinimuin slAndard of 
literary iiilrrpreli(T^n and writing. 

The g! owing inti rc^t in cultural exchange and comparative education 
OMv M»nie day make pi»^sil Ic ihe administrati<)n of suilaMe tests, Itoth 
\)\ liiis rnuotiy and in Hussja, in groups whose picparation has been 
ciiniparahh'. Il wtadd l>e |iarliculnrly illuminating to see if there arc 
>i;;nilicanl dilleicnccs in the performance of American a'ul Russian 
sfu.lf'Ol ^ and, i( h,>. luiw they difler. On the hasis i»f [ escnt knowledge, 
nor nu,:litjia/ait| the guess that in cunndative sni jet s sncli as ma'he. 
lu.iiH Inif i^n la»iguagr. and ihv sciences the |{u**sian sUidt'Ols will havr^ 
,1 I rlin i nnnnathl nf minimum (^s^entiaN> wlicrcas American sUidc nis 
\sdl \u\\r \ l.n adrr view nf the suhjtM't and perhaps s<»mewhat more 
III xil d»ly in ajiplyujg the knowledge they^ have gaine<l in n variety of 
situations. 

\ j 
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Much of thr rdunition in the Sovit^t Union at the present time seems 
(HM uhar tn uh. Many a^pn tn we would ciilirize, Neverthclrss, we have 
io rt'Citgni/c (hut hrtr a fnuntry that han inaJr tremendous Mtrides in 
oiluralinn liuuii^ thf past forty yrarn. Tlie peopK» have had to overcome 
great tihstach'H. They have heen tackling their problems with imagination 
arul they are cxperinirutally'minded. They are flexihle and adaptahle; 
they are enerp^tir in putting ohungen into effect on a nationwide basin; 
and nio^t iniportarit oi ;ill, tht^ havi* a treinrndoun hehef in education. 
They will presH furwar*! in the educational field— not to make life full 
for the indiviilual, not to bring about \m full developmtMU so that he will 
Irad a rich and rt wanling life, but to Tuake a strong, powerful country 
to reach and surpass Anirrica cientifically, niihtariiy, and economically. 
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Session III 



Heniarks of the Chairman 



We rome now lo the final Mejision' of this Conferenre, aud something of a 
charge of pace aH nell a» a change in topie. Thit» nesHion will b« a panel 
(liAcusflion on one sul jeet with four difrerrnt.iipeakerfl presenting diveme 
viewR on the problem pimed. 

We havp all been hearing, until the expression has become a cliche, 
of the imptMiding tidal wave of college students. What the tide is going 
to be, high tide or low tide, is going to depend in considerable part upon 
the selection and admissions procedur(*s that are adopted by our 
institutions of higher learning. 

The determination of the optimum types of examinations for college 
admission and scholarship purposes is certainly one of the most pressing 
problems in the measurement field. We have brought together this after* 
noon, for a thorough airing' of the issues it^olved in this problem, four 
prrsons who have devoted long and careful thought to these matters, and 
who have engaged in extensive research on the topic, t am confident they 
Ivill cover the field with comprehensiveness aijd that they will bring to 
your attention the pros and cons of the varying point** of view on the 
testing neefis in this area. 

It is a pleasure to introduce to you our four panel members: Dr. Robert 
L Ebel, Vice President of the Educational Testing Service, will lead off 
with his views on this afternoon^s big question, '' what Kinds of Tests 
for College Admission and Scholarship Programs?'* The second panel 
speaker is Dr. John C. Managan of the American Institute for Research 
and the University of Pittsburgh, who will Aisbuss ''Criteria (ot Selecting 
Teftis Jor College Admissions and Scholarship Programs.'' 

Next will come Dr. £. P. Lindquist, of the State University of Iowa, 
who will focus his remarks on 'The Nature of the Problem of Improving 
Scholarship and College Entrance Examinations." And our final speaker 
is Dr. Alexander 0. Wesman, of The Psychological Corporation, who will 
auminaritp^ ua ^is answer to the question, **What Kinds of Tests for 
•College Admission and Scholarship ProgiSknt?'' 




What Kinds of Tests for Colle-je 
Admission and Scholarship Programs? 



Robert L Ebel^ Vice President, Educational Testing Service 



The uho oi ti*Ht^ for college admission and Acholarnhip programi has 
grown rapiiily in rrornt years^ and seems likely to continue to grow 
\i\ the years immeJiately ahead. If soi the wisdo«n and experience of 
you who are here twJay will have an important bearing on *he future ef- 
fertivenesA of Ameriean higher education. I an^ grateful for the oppor- 
tunity of taking part in the discussion of one problem associated with 
this development, and am honored to be included in so distinguished a 
panH. Y()u will uuderstand* I am sure, that on an occasion such as this* 
I will try to exprrs'4 my own vinws as frankly and clearly as possible^ 
anil that I will not presume to speak for ETS or any other institution 
or group* 

What kinds of tests should be used for college admission and scholar- 
ship programs? The answer to this question is, in principle, quite 
simple. We should use the tests which are the most valid. Some people 
think the answer, in praetire, is equally simple. Choose the test or 
combination of tests which gives the best prediction of first year 
college grades. 1 demur. It seem^to me that this approach is not likely 
to yield adequate evidence concerning the relative merits of different 
kinds of tests. 

There are at least three limitations of conventional validity studies 
in this situation, as I see it. First, the criterion of college gn ^es or 
grade point averages is itself unreliable and of quite imperfect validity. 
Second, sampling errors associated with the obtained validity co« 
efficients tend to be so large, and precise estimates of those sampling 
errors so difficult to achieve; that it is almost impossible to make de* 
pendable comparisons of the merits of alternative tests. Third, and most 
important, this approach assumes that a collegers current educational 
program is beyond criticism or improvement. If students who score 
high on the selection test do poorly in college, the selection test rather 
than the college program is bl. ^d. I submit that there are better 
ways of improving the input to our colleges than by striving tp improv* 
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the prrilictidii ol iaulty mraHures ofstudt nt AUccctts in attaining poorly 
driinni ami nomrwhat qurstionabit* goals. 

One of the»e ^br drscrihrd hy Ralph Tylf r» writing on ''Educability 
and thr Schools** in the Centennial volume^of the American Aasociation 
for the Advancement of Science, He contrasted the predictidn-of-succesi 
approach with a dcvelopment-of talent approach that woiild leek to 
capitalize on the important latem ahilitiea revealed by appropriate tests. 
He suggested that our present srHool programs do not capitalize on all 
Mucn abilities, and that tests designed only to predict success in current 
programs of instruction do not adequately measure the characteristics 
which determine educubility. I heartily agree. 

There are some who mii'trust subjective decisions concerning the 
natureM>f these <'haracteri^tics» preferring tcqts which can be defended 
on the l)a9iH of their high correlation with some ultimate and presumably 
more objectively given criterion. But th^ only truly ultimate criterion 
of success if indeed there is any such thing, lies hidden beyond the 
grave. And the apparent objectivity of some criteria hides the subjec- 
tivity of our decinion to use them as criteria. What reason have we to 
believe that our subjective decision to use more immediate criteiia of 
8urr '!4H, such an wealth, or grades in the- freshman year of college, are 
any more tru^^twortliy than mir subjective judgments concerning the ele- 
nientH of a good preparatiou for higher cdueatitm? 

Kven if we are willing to overlook the possible imperfections of our 
( riteria and experimental designs, why should we employ an approach 
which allows the inevitable contingencies affecting success^-health, 
fniancei^, motivation, eve .1 romoncc— to blur and becloud the application 
of these judgments to the choice of admission or' scholarship tests. 
Would it not be better tg apply the inescapable acts of judgment to the 
tests directly? And is this not actually what we do? Were not most of the 
college admis!«ion and scholarship tests in current use designed*and built 
on the lui^is of rutii'nul irduetjions, deducti(ms and hypotheses? Empirical 
prnrediircH ftuielv wne u-^ed lo defend them, and'to refine them, but the 
basic structure seems usually to have been determined, as I think it 
should ba>e been, by purposeful cogitation rather than by completely ')b- 
jective, judgment-fi. experimentation. Hence, what I have said about 
the limitations ol the grade criterion and the conventional validity study 
fi^hould not be eonstrfit d as u complete rejection of such studies. Some of 
our judgments al o^t what we want to measure, and what we have sue- 
eeedf (I in nuMMn inf;, can lie eherkc d empiric ally. l)ut I am convinced that 
validity studies should not be the exclusive, 01 even the primary, basis 
for test selecMion. One test or battery of tests should not be chosen over 
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another nirnply hrruuno of a Hmall apparent advantage in predictive 
validity. 

Among the va!«t array of te»ts wliith might be used in admiaaion and 
Mch'olarnhip programs three major types c4in be identified: testa of innate 
rapacity for learning, of (levtloped ability, and tests of.substantive 
knowledf^e. * 

Miioy |HM)ph* ngard ivsx^ of ionale capacity for learning an nearly 
idrui fur college admission and scholarship programs. The term innate^ 
mental capacity suggests something which is fundamental and permanent, 
and hem*e well worth measuring, a divinity that shapes our ends, no 
mutter how poverty, inadequate schools, ot youthful follies may have 
rough*hewed them. A test of innate capacity presuiiiably will not handi- 
ra|> the bright youth who grew up on the wrong side of the tracks. In 
thf ory/ scoring on it should not be affected appreciably by coaching, or 
ihilced by instructiiin of any sort. Such a test would place no restrictions 
upon the secondare school curriculum. Local schools and individual 
it achers could presumably retain' their freedom to teach what they 
clioone, and it would make no difference if they taught it well or badly. 

Unfortunately for this particular vision of uto|[>ia, no one hu found 
any very accurate way to measure innate capacity for learning. All of 
tlic alleged measures of this capacity are more or less obvious measures 
nt rdui ational achievement. The main differences among them arise 
tium the vkvying degrefs of earnestness with whi<^h their authors have 
attempted to avoid measuring the results of school learning. The kinds 
i>r tunkn a studetit is asked to perform in taking an intejiigence test are 
taHk> which he has l)een taught to perform in school, or could be taught 
(here if the schofd considered them of siifTKMent importance. Lacking 
prt cise control of educational influencci^, pre-school, in«school, Jind 
i>ot of*schooL we cannot tell how much of a studefit^a success on an 
intelligence test nhould be attributed to his native mental capacity,' 
anil how much io his subse^f«ifnt learning. To regard scores on such tests 
SA acceptable measures of innate capacity for learning requires as^ump- 
whicli I find, hard to accept. 

Kven if tejitr :»f innate capacity for learning were available, they 
probally ^ould not be desirable for college admission and scholarship 
programs. Innate capacity is not directly relevant to ability to profit 
from college instruction. I tJess this innate capacity has been developed 
—unless the studcni has acquired considerable knowledge and numerous 
abilities<-*-he ts \inprr'«f)are(l for college no matter how large his empty 
innate capac ity may have been. • 

Some frar thM unless we u^e intelligence tests we may miss brfght 
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HtudfiiU who, growing up in an rducalinnally deprived f nvinmnifnt, 
dhow little achievement and hence might be denied an opportunity for 
rollege education. Evidence which would justify this fear is hard to find. 
Many caaes of brilliant adults who were unpromising youthful scholars 
can be ( ited. But efforts to identify such individuals in advance by using 
tests of mental capacity have been disappointing. Errors of measurement 
can account for much of the observed discrepancy between measures of 
so-railed ability and achievement Differences in the kind of achievement « 
required by the two types of testa can account for much more. Aware of 
these facta, most of us have ceased to trust achievement' quotients, but 
some perpetuate the same fallacies searching for over- and under- 
achievers. Some **under-achievers" go rn to college success. So do some 
over^achievers. Intelligence tests do not help very much in identifying 
brilliant minds which the schools have missed, or failed to develop. 
All learning builds on previous learning. If the foundation is weak the 
superstructure is likely to suffer. What the colleges require are students 
nho have strong education foundalionf*, not ()u)>c jHisr^rssiiig lirilliaiit 
but undrvrlopVd miinN. 

ihw (illtTiKUivc III inra>iuf mriit ol iirtlivr r.MMilul capacity u hiisi^ * 
lor rollrgr iiiliniH««ioii i\\u\ s(*h(ilar?thip uKiir(l> is ihr meaHurcmnit of a 
s(ndnit*s roniniaiid rnsriilial kiiowtnlgt*. Thi?^ allrrnalivc ban not brcn 
popular in rrrnit yrars. kiiowlnlgc is coiiirastrd with ignnraiUM\ 

it is uiiivrrsnllv praised. Kiil kfiowledf^e in Kunirlinirs cnnlra^lcd witli 
undrr^^taiiiliii^, with vvinjofn. or nith rliaracttM'i aiul iii tluv^^ejontra^tH 
knowinlgr Uwrn Itailly. 'PrarhrrH sprak diAparagiiigly ol "»tu(Iing the 
mind with fart*^/' (!n||t*gr prt*!(ii1ont» rniphani/r the limitations of **mcrr 
kftowlrd^t*/' and nhrss thr cofitriliiilioii.H thai a rollt*gr education can 
make t(» the dt*V(*htpmrfit of rhaiarttM'. Quiz kidn, or cvcu adultn wlm 
slmw ari iiiHisiial ^copr and accuracy ifi their m all of isolated items of 
iiilortnatioii, arr u A to jlluHtralc what a go6d rdiiration does not 
vo\)s'\^\ of. ''Srraf)H h1 inioi iti.itiou have nothing to do with riillurr/^ 
Whitrhrad ha«( ^lid ami oMlimicd, mrrrly wril-infornicd man i» the 
nio«*t iisch'ss hnrr (hmTm rarth/' 

Attarks like thcsr oo pcdanlir, trivial, vrrhalislir, unasHimilatrd 
knnwlrdgt* have hern at Irast partly rcspoMHihlr for general reluctance to 
use IchIh (»t •^nhslanlivr knowledge in college admin^io?! tci^ting programs. 
Rut I wouhl suggest that this poliey deserves rei^xamifmtion. Quite ob* 
viiMinly, knowledge ran he defituMi so narrowly, or caricatured so gro* 
lesipiely, that all of the ahov<' attarks on it will seern to W well founded. 
Hut knnwirdye wrrA ?iot be limited to isolated, trivial, informational 
dctaiU, nor to verbal ahstraetions divorced from reality, nor to rote 
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ret^poiiHoA to Htorfotyped quentions. It can be defined broadly enough to 
include undrrslundinn, wisdom, and o|hcr approbatory synonyms for 
I'fTrctive rational, behavior. Understauchng, for example, consists in 
knowing the interrelalions between other ittms of knowledge.' Wisdom 
i'on8i»t9 of knowing how to use ihelnowledge one possesses. Character, 
insofar as it is expressed in rational rather than blindly imitative or 



knowledge of the 
this broader con* 
and argue that it 
process. Quoting 
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authoritativ(*ly imposed behaviori is based partly on 
consequences of altrrnative courses of action. It is in 
text that one can truly say that knowledge is power, 
represents the primary oulcome of the educational 
Whitehead aguin, ^'tAiuration is the acquisition of the 
tioii of knowledge/* I suggest that the "art** Whitehead speaks of is 
itself essentially knowledge of how best to proceed in a given set of 
circumj^lances. 

Apart from unwarranted restrictions on the concept of knowledge, 
there is another reason why knowledge tests have not been widely used 
as a basi-^ for college admission tcsliiig. The scope of human knowledge is 
so broad, and (he areas with which difTerent individuals are acquainted 
are so diverse tliat Ti seems difficult to qonstruct any single test, or 
even any limited ballery of tests, which can deal adequaTely with this 
abundance and diversity. Our decentralized school syslem, and pupil* 
centered leaching procedures tend to foster wide differences in the kind 
and level of education lhal different pupils receive. Buy since we live in 
the same society at the same period in history, all of^s need to know 
and to be able to do many of the same things. It mayN^e difficult but 
it is not impossible to define a common core of essentyl knowledge* 

There is a third alternative to tests of menUl capacity, and tests of 
command of essential knowledge, for college admission and scholarship 
testing. These are tests of mental traits or developed abilities. Proponents 
of tests oi this lype refer to thvm as measures of broad intellectual 
skills, of liasic aiental processes, or of habits of thinking. It is said, in 
their behalf* »hat they measure what a student is able to do, more or less 
independeiui of what he knows. The abilities they purport to mclsure 
range from the higlily general, such as "abfli^y to think** to the fairly 
specific, such as *'ttbihty to formulate hypotheses.** 

Insofar as these tests emphasize essential rather than trivial achieve* 
r.ent, rec^uire the student to show understanding rather than mere 
recall, and ability to use rather than mere ability to rept jducc, they 
deserve enthusiastic applause. But when they are represented as measures 
of distinct mental processes or iialcpendent intellectual skills, something 
different from and more important than knowledge, which can bedcvel* 
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opnl hy prurlire with a wide varif ly of content mate rials, they aefin to 
^ call for critical rxamrnation. ^ 

One thing abiiut them which is troublesome is the vagueness of what 
they re^er to* There is no lack of names for alleged mental Abilities, 
but there is almoHt universal lack of sp< cific definition of what these . 
names m^'an. The evidence that these abilities are independent of . 
knowledge, repre^^rnting a kind of mental ianction which can be applied 
at will to knowledge in diverse areas, is practically* non-existent. They 
suggest a renaissance of the once discredited belief in faculty psychology 
which hrM, for example, that memory for something like faces could be; 
improved by practice in remembering something else like spelling words. 

If proponents of the developed abilities approach to mental measure? 
ment regard such things as ability to calculate a square root, or tO 
diagram a sentence, or to locate a malfunclioning element in a television 
I. tMrrnit, as developed abilities, I have no quarrel with them. We are 
simply nsin^; difTerent words for the same thing when they call them 
dtweloprd al>ilities and I call them command of substantive ^knowledge! 
lUit when proponents of tests of developed abilities spe ^* of highly 
giMiei alized, undefined or vaguely defnied complex higher mental processes 
or mental traits, we part company. By using high-sounding terms wc may 
impress outsiders but we are not likely to contribute much to the 
improvetqcnt of nn'ntal measuremejits. 

Many o{ the so-called "hard-t^-measure** qualities appear td ^atl 
ill this category of developed abilities. Few would deny that terms like 
"creativity,** "(lexibibty,*' "sensitivity," or "balanced judgment" are 
useful ill describing behavior.' But we make a tremendous logical leap 
when we assume that they are also names for mental traits which help to 
determine the behavior observed. Before agieeing that our failure to 
measure these "trails" is a blemish on the record of mental measure* 
incnts, I wnubl like to be a little more rtain that sometbiiif; exists to be 
measured. Skiimer, Holland and others have pointed out that p^ychob 
ogists have a weakness fc»r inventing explanatory concepts to aocount 
lor meager or non-existent data. Perhaps this tendency, which many 
educators also sliare, is evidence for a Iiard-to-measure mental trait 
called "lialhicinat(»ry imaginativity." ' 

The developed abilities approach to the measurement of educational 
aptitude is based on an allegoriqilly attractive, but experimentally 
unsubstantiated conception of how the mind functions. All that wc have 
learned concerning the process of learning in vats, monkeys, apd men, 
can be explained in terms of the formfition and destruction of assdciations 
among perceptions, concepts and ideas. Electronic braini, including 
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primitive mo<iel» capable of learning or even of self*TeproduGti( 
operate ettsentially un the ^ia of nwitchec which open or clote alterna- 
tive circuits. To regard the mind a» a muscle which it itrengthened by 
ex#rci8e« or a)) a procetsing organ which can be trained to ate relation- 
»hipa, to solve problems, to analyze, to think criticallyv or to create, 
irrenpcrtive of what kin<l of problems are given to be aolvedf what ma- 
terials to analyse or think critically ab^t, or what product to be created, 
Hinipiy does not accord with what we now know about brain functions. 
What wf do know strongly suggests that educational development 
con<iists essentially in the accumulation, integration^ and ready-reference- 
indexing, of knowlitdge. By pra< tice in using knowledge, one acquires 
command of it. This, it seems tn n^e,' is the essence of educational 
achievement. 

Tests of developed ability arc sometimes offered' as a method fur* 
keeping peace between test constructors on the one hand, and curriculum 
builders, school administrators nnd ^eacbei:^ on the other. For it is 
suggested that tests of this typo measure how well a student has been 
educated without regard for anything specific that he may or may not 
have, learned. This, it seems to me, involves a contradiction, since the 
(]U{?litY of a person^ edocation cannot bo independent of what he has 
learned. Any lej<l Hhieb disniniinates belween well and poorly educated 
^tudent^1 will inevitahly reward the curri<!ulum builders, scho<il <iflieials 
and teachers wbo have been nif»^t sueccs>^ful in imparting a good educa* 
tion. Of course a test wbieb n wards those students who have learned 
^oine arbitrary selection of unimportant details is not a good test of 
quality oriMlucatinn. To quahfy for this designation, it must measure the 
degree to whieb a sUideiit has achieved command of the most important 
general ideas and skills. Il is not easy to reach agreement on what the 
most important ideai^ and skills are, but we cannot tnake good tests if we 
try to dodge the prol)leni. And we should not mislead test users into 
thinking tbat we can. 

Tr^{ builders obv^ou^ly sliould not dominate tbe curriculum or 
dictate what tcacbi rs sboMid teaclh But 8ome<me must define common 
edueatiotiat goats ^^pecilically etiough to permit deterrtiination of the 
• xtent to wliit b tliey arc being reached. In this endeavor the test con- 
structors can be essential allies of curriculum builders, school adminis- 
trators and teachers, especially if they concentrate on measuring a 
student's command of essential knowledge. 

It is quite apparent tbat teots of developed abilities do not actually 
Hucreed in rAtusuring mental processes apart from the examinee's 
, knowledge or lack ol kfuiwledge of particular items of information. They 
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do int'Oture Liiuwl(*dgt% but oiiiee thoy are deigned to measure some- 
thing eUe, tht^y often provide piiorly balanced, inefficient tesU of 
knowteiige. Tlie a.^^sumed necessity of supplying essential background 
^information in the teHt Ui^elf leads to relatively verbose tests, which give 
undue wc^ight to reading t^kills, and require undue amounts of time per 
Hcorable renponse. Philip Verniui suggests that tests of this type may 
yield a .est form factor nhich seriously biases the scores from such tests. 
The score a student iibtains may depend considerably on his knowledge 
of how to handle the particular type of task these tests present. Further, 
the use (»f background materials leads to tests composed of clusters of 
items, wliich restricts the freedom of sampling in the test and complicates 
the prticcHses of item selection. Hence I am persuaded that a well- 
desi({ued test of substantive knowledge which measures a student^s 
command of Imsic 'useful knowledge will provide more relevant infor- 
mation concerning a student's educational achievement and aptitude,^ 
and provide it more efficiently, than current tests of educational 
development or developed abilities. 

Although these tests «>f complex mental processes and developed 
general al)ilities have been in use for more than a decade, there is little 
evidence thai thi^y measure something more important than can be 
measured by ti sts of n student's command of sulmtanlive )0^wledg6i 
Some of tlit ir advocates argue that such evidence is unobtainable in 
principle, and thut the only Hiiy to ascertain their superiority is by 
looking at tht* t(^nts thi msclves. But the main job of a test is tp yield 
useful ^t'nrct^ from examihcrs. If the scores from one test are better than 
those frnm aunther, they must at least be c^iabiv dilTerent. If the correla- 
tion of St llrc^ on a superior and an inferior test js as high asHhat between 
iwn form^i of the superi<»r lest, then it seems to me that ail of the claimed 
su|>efiofity is geUing tftst iti the errtTfs nf measurementv 1 do not see 
how such a lest can Im^ said Ut be superior functionally as measur- 
iuf{ instrument. 

A dr!*lral»le characteristic of i college admission testing program' is 
that it liave a stimuluting, constructive influence on the programs ol 
instruction in pfeparutory schools. Obviously a test/if mental capacity 
could not have this innnenc(\ Tests of developed ability because of the 
generality am! indefniiteness of what they are testing, are uiiUkeU to 
stimulate either teachers or students to unusual elTorts to improve. 
Since the first Sputnik orbited in space, the practices of llie Russian 
schools liave reecivetl considerable attentioti'. It would be foolish to 
sugg(\st that we shogld copy exactly wliat they are doing. But it would ^le 
equally fooli^li for us to ignore completely their accfmiplishments, and 
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\he means by whiciv thpi* were achieved. Henry Chaiin.^ey and others 
who liavr visilrd Rtls^ian 5chooU report wilh some adniiralion the 
persiHirnt nrul generally efToclive efTorta of teaelierft anil students to 
achieve limited but clearly delined goals in achieving command df 
substantive kn«)wledge. If we wish our college admission and scholarship 
award programs to have the most beneficial effect in the improvement of 
public education, it would seem wian to emphasize tests in which the 
n^aure, and above atl the quality, of the leaching done by each teacher 
makes a direct and obvioui difference in the score:) tlie students receive. 

I suspect that m'y cjillcagues on this panel^Jo,not share, completely 
my admiration for tests of subsUmtive knowledge. If it were possible to 
do so, I would like to test our dlyergcnt views^with a little experiment 
in which we w^nild compete wilh each otiier in seftcling capable students, 
J>uppt)se that one of us prefers a test of mental capacity, another some 
tests of developed ability, and yet another a battery of pure factor tests. 
My preference, i f course, would be for a ter^t of relevant Bubstanlive 
kn(»wledge.'Sop|iose that we have available to us a populaliJfi of lOf) 
pupils who are just ready to be taught some new process; sulh as the 
soUiti(tn of simultaneous linear equations in algebni. Assume that each 
of us is a tolerable teacher of the process/llie .students are about to 
le;irn. We each give our preferred aptitude' test to all of the students, 
and {\\v\^ proceed to choose up classes. After clioosing our classes* and 
after an agreed-upon number of periods of instruciron, we would give our 
students a c(Mnnu)n test of achievement and dcteVmine wliich of uj had 
done the best job of aeleeling students, and of leaching lliem. 

This is a hypothetical experiment, not only because we are unlikely 
to have an opportunity to perform it, but also because I seriously doMbt 
that any of the other panel members would wish to rely, when the chips 
are down, on a lest of menial capacity, developed ability, or pure factrm^i 
to scleK sluilcnls wilh the, greatefjt iiptilude' for a particular job 'of 
learning. I suspect we might all agree that in this situation a specific test 
(if sttbslaiitivc knowledge would be more clTnclive in selecting students 
than a general lest of capacity, of dcvebiped abilities, or of mental 
fa4 li»rs. My colleagues may object that I have prejudiced the argument 
in my favi»r by directing attention to a sin^ile specific problem of learning 
and teaching rather than on the diversity of such problems which face 
the sludenl admilled to college, and bis teachc-s. This is a reasonable 
objcclion to drawing general conclusions from the experiment I pro- 
posed, but It has interesting implications. It suggests that tests of 
menial capacity or developed abilities should be used, not because they 
measure something more basic to effective learning than acquired knowl- 
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nlp\ hut lnMiUHf they prdvido coiivniifiit general measures of eom- 
pdrnrc. tlit y tin, hut I atii iiieliiiril in rinuht it. Unless I um 

Hii^lakrn ahnui tlir ph.inlntn nature of many mental traits and ileveloped 
ahilitirs, athi Ihr ^lili-ilaulive knnwl«'ilp» luiture ol' the rest, tlu» lest way 
of niea iiiin^ a ^tn»lfnt'| |»re|Kiratii)U fnr enllrf^e learning in genrnil 
is to sri (Mil dirrrilv In ni^asure ihe most grnerally useful nj^prels of 
»uh'^l.inli\r kimwlril^r. 

TluH alternoon 1 have (|ue^tioned some widely ludd opinions almut 
n|ailude Irsis, If vsluil I have said sounds do(;nuitie, the reason is nt 
Ua^t parlly lhat ynu and I have not yet done enough researeh on funda* 
menial ptohh ins ni a|)lilnde testing to provide a more suhstantial basis 
lor our helirf-. 1 am Far from helieviuf^ thai the views I have -expressed 
arr the otdy ours any reasonahle man ean (Miterlain. In faet it is now 
linn* for m^: to yji lil to another reasonahle man wh(» will present soni^ 
othfr virw^ for vou lo eonsider. 
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riir U\ u ;is^uin|ilinn lujilorlying this «li.Hrussinu i.s llial in oyn fuunliy 
0\r rnlln tivt' aim is to cnalilf rucli imliviilual Ut rrali/<* liis Inglir^t 
jM^irnlial. This riinira>lH ^fiarply Willi llir Russia view lluil llu* aim nf 
r«lnralior» (and all uiIht arlivilirs) is lf» nvdVv llir rnunlry a» pownlul 
a> po-ssiMr. 

Siiu't' imiivitlual ral«MU> ran»:ul Im' cxprrltNl tn Jrvrlnji prtiprrly in iiur 
<'.nnplr\ MM'ii ly williniil sysh nialir, appioprial«\ ami t'Xlt-nsivr nnrlurr, 
11 is uf ulniosl impiM'laniM* llial ihr imlividuaPs lali-nls l.r iflrnlifuMl 
rally as a !>asis Tor lii> rduralumal and rarrrr plan?^. This poiril tif vi«*w 
•.M^rsl.H ihal rolli*^;i» adniissiuus shoultl lir virwrd as a pari o( an 
Dvcrall sUulrtil rnuMM*lin^ ami ^iiiiiuiKo pruf^ram. 

It follows llial n lli'f;rs and univrrsilirs and nlhrr organizations 
-poim(»riMg »rholar>lnp programs miglil most appropriaUdy eidisl ihr 
.lid of routi«tdors, srrondary ftrhool IraiditTs, and otIuT schocd odirials 
Ml ansisting thrm in drtrrrniriing whirli sluilrnlH hIiouKI purj^ur thrir 
■ diH' iM'ii uridfT tlir auspirrs of carh fd thrs*' collrgrs, Tlirn* arr .wrral 
t urn .' . .Ual roiisidfTations wiiirh haw ' otilribu^rd to this rorit luftioii. 
Ka<di . / tlirs(^ will discussed liiirliy. 

I. /I rlmr Mntemvnt of ihv objevtivvs of xoUvgr, univrrsity, or cthn 
or^ttnizotion should he the uUimote hasii^ for ra^ef^e admisvons <\nd fnhobir- 
ship polit ies, Unlrss ihe institution knows pirciscdy wlial it wislujft to 
a« • Dmplisli, tlirrr ran In* no rvuluation of it^ j^urcess or failurr, For 
tins purposr, a g»'m»ral statrmrnt of objrrrivrs will not ftufTicr, The 
Im nad iiini!^ of th/ institution mu8t Im- tianslutrd vividly and with ifrlaih'd 
iwatMuIrs if tlir) Mrr to providr a prartical framrwork for developing 
srliTVioii pidirirs. 

Of n»ursr, hriuid MtatrimMils n{ aim arr tirnlrd a« a lia^i.^ for dr 
vrloping sprrilir ainn. A good exampir i-i tin roM d mm ipioled frtuii 
irmnii^ of \lr pnwidriit of onr «d our Iniding nnivrrj^itir^. In dift- 
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niHHing tho olijtM^tivrH of a rollego administrator, he proposed that the 
aim should Ik* . . to hrip i*vrry young person in his care grow into the 
broadest, dre|>est, most vital person possible. And in fulfilling himself, 
the student will . . . arrive ut moments of heightened insight when he 
seeH more elearly than ever before what the world is about and how he 
ean fit into it rreativrly and significantly/* 

Sneli a general aim muHt be translated into specific activities of 
faculty and stuiliMits if it is to be useful for formulating student selec* 
titm procedures. Detailed statements about the dynamics of such student 
growth can contribute directly to decisions regarding selection policies 
and procedures. 

Insofar as colleges reject the objective of **just cramming students 
with facts in order to teach them how to earn a living/* college grades 
and a(*hievement in typical college achievement tests cannot be regarded 
as a satisfactory criterion with which to evaluate selection procedures. 
The (!o||cgeH must do something about developing working statements 
of their objectives if researcli answers are to be obtained. 

2. Si'holnrship and vollege ndmission po^h ' 5 might well he regarded as 
an integral part of a broad program of ind.jidual guidance. It is proposed 
that, lacking the detailed statements of college objectives referred to 
above, this aim be defined us '^to make it possible for each young person 
to identify and obtain the education necessary for him to realize his in- 
itividoul potentialities and to gain lusting personal satisfactions.*' 

The [primary implications (d this consideraticm are that the guidance 
(Miunselors and the college admissions ofTicers should regard themsi;!ves 
as u team working* together tu at^liicve the objectives of both the in- 
dividuals ami the institutions to the greatest extent posfible. In order 
for this ti*am to rundiori eiTectivcly, colb*ge admissions oflicers should 
t'omnutnicate di'taib^l inlormation to the secondary 8chool counselors 
regarding ihe s|mm ilir opportunities fwr educational development at 
their institutions. This inrormatiori should include reports concerning 
the cbararteristii's of the studrnls wlii» benefit most from these oppor- 
tunities. 

Tlic finiction of the <'ounselor is to collect and communicate data 
regarding individual students of the types identified by college officials. 
rsych(»logists and iMhtrationul measurement specialists should make 
t'vcry cfTorl to ilcvrlnp satisfactory tests and reljited procedures to aid 
eonns(*lors in this task. However, in the absence of satisfactory psycho- 
metric tiM'hni<pics« roiinsrlors should use all informal data gathering 
procedures available to provide as gonti an t^stimate us possible of the 
students' cbiuarleri-^lii s. 
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3. (lonnislt^nl wilh i\\v lirst tW(> poiiitH, Mr primary rritfrion for evaluat- 
ing M'lectiun fu)liries and pnxrilures i.i the performance of the individual 
after he leaver the college, Tliih HtatorntMU do^ji not mean to imply lhal 
(M*rf(»rnmncf- in an nrrupalioii in to be taken as the only criterion. It is 
intended that performanre with respect to all aspects of the life of an 
educated individual ineluding citi^ennhip, parenthood, cultural and 
personal development he included. This focus on life rather than on 
schooling for evaluating admissions policies casts further doubt on the 
adequacy of current procedures. 

Two recent studies, for example, show little relation between pre- 
dictions l>ased on ability and achievement and subsequent performance. 
The first of these is Terman and Oden^s follow-up study of gifted children. 
In this study the conspicuously successful group and the relatively less 
successful group showed only a slight difference in average intelligence 
test scores. The other study has been reported by Harmon. This consists 
of a follow-up of a group of applicants for scholarships. Committees of 
professors usin^ college records, recommendations, and other infor- 
mation selected some of tlic group for scholarships and rejected others. 
It is reported that several years later, at the time of the follow-up, the 
average performance after graduation of the individuals in the two 
groups was nearly indiHlinguishable, according to the appraisals made 
by new committers. These new committees consisted of persons of the 
same type as bad made the original appraisals. In this study, grade point 
averages and tests of verbal ainlity, quantitative ability, and educational 
achievement all bowed predictive validities of approximately zero. 
It seems itiappropr ate to continue to rely on these general measures. 

4. The pattern of aptitudes required for sua ess in each of the important 
career fields M relatively specific to that field. B<*cause of the dependence of 
educutituial iustrnctii»n on verbal comprelieiision and the ability to 
solve quantitative protilcms, it lias been ussunHnl that these abilities are 
the primary determinants of snrccssfnl performance in nearly all the 
career \\v\\U for wliii h college training is provided. One o( the early 
refutations ot itiin point of vi(*w was the successful discrimination 
betwcfMi the atititodi* [latterns required for successful performance of 
pilots, navigaloTH, and other aircrew members in »be Air Force during 
World War II. In I'Ml, ofli<ers respornible for selecting pilot* were 
Hcb cMng applirnnts on tlicir knowledge of history, literature, ability to 
read, and vocabulary. Some of these niea:4nres were found to have slight 
negalivc corrclalionH with .-^ocress in pilot traiv ng. The widely bcM 
view thai ;i pcrsoti nilli average ability iti dcalii;g with verbal and quan- 
lilaliv*' tnaleriaU ran j^ucccimI in praclically anylliing if be merely 
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ii(»plir!4 himHririj,gr;iilually ln-ing n |.liirc'(l Uy thr morf j^opliislifulnl vifw 
tliut a larfjc nuinh(*r ofsprrilic nplitudt's play irnpnrlanl rolt-s in flcUTinin- 
itig pnrMriiiaiir** in [\\v iinpdrlant (•arrt»r fii^Ids. 

r>. Intf'rrst uml mndvntion fnrUtrs arc at least CfjiuU iu ifuporUmre to 
a/itittuir fiirtors in drtrrminin^ fmfomana* in spevijic varver fields. It is 
wvW vi*U\\Miii\ that tlw* lop pi rfuniKTH in many fwlU are ihusr inohl 
-Irnn^Iy imoIivjIimI »itli rr^prrl U\ llmt nrtivily. I'litM-norl and piTsislrnrr 
wliirh arr ol»>rrvn| ti' ai rcinipiiny nntstandiu^ rfTtM livt'ni'ss an* haml on 
.1 Ingh Irvi'l of int«Tr^l. In {\\v Army Air Fonr during World War II, 
il wa:A lonnd that tin* Ur-^{ Hin^;li» prrdiiior of sucrrHs in p:!*>» 'raining was 
a mrajiurf ol iiiii'ivsi and nu)livali<»n in the orin of ai. infornialion Irsl. 

A iprridi- rxainplr of the rflVi ls uf insnHirirnl inlrrcsl and motivation 
is pnnidrd J*n ;ni Amcri<Mn pilol wln» was tt-strd as An tMnployiT oi' a 
foreign air linr. This pil(»t had hvrw Irainrd dnring Worhl War II in ihr 
I!. .S. Army Air Kon r. AftiT lh»' \>ar hr w»»nl hack inl(» iht' dt^parlmt'nl 
Hlorr Im^inr-^. HowfM'r, hrrausr of [\\v largf salari»»s availahir in 
rouuni'rrial living, he airrplfd an oWvr to Hy for i\\v forrign air linr. 
His aptitiidrs wvrv found to ht* rrlativr'ly high with rt^sprrl to all of ihr 
ahihlir-. r<ipiiie'd i«| air hnt* pih»ts. On llu* trsis having t<» do with 
inlrrr^t and nioti\atimK on thi- oIIum hand, his sror«»s unusually 
l.)W, liis paltrrn (d* inlt rtwi^; \>iis strongest in husinrss, arrounling, 
litrraluri', and llu- fine arts, lie showed ainuist no interest in meehanioal 
pr4»hletTM in general or aviation matters in partienlar. 

The i hief pilot ol' thf air lint* reported that the pilot in cjuestion 
spent niosl of hi^ lime reading poetry or elassieal literature while his 
eo^pilol llrv> the airplane, lie made no elTint tr> h»arn ahoitt new onuip. 
mrtW or drvirr-., \>v to maintain liis flying skills, 'fhey rep<\Vted that in 
lln» tNM> year- he had heen Hying for their air lin«»ln» had tvvi(!i» failed to 
pass his six. month inMrnnuMit ehet k. luieh lime he had heen taken ofTfly- 
ing and given >pt ('ial training \u\c\\ time he re.sponrled vrTy qniekly to 
the speeial training and was soon ahle lo pass Ins eheek flight and go 
hark on Hying ^tatos. CIrarly, sneh an individual is iniseasl as a pilol 
and is not nnly inelfretive, hnt ii [tolrtilial ha/ar<l to eompan) e(|uipmrnt 
am! pa--f nger lives. Many in^tanees of the reverse situation in whiidi a 
high degi»e of inte'rest and mnii\ation have nn»re than made up U)r 
^pt'i'ilir aptilndf defieiein ir- have also hren ol^^elved. 

0. .i owtfurhmsivr inofnam nf trsnnv/i is irtjuitnl to it/mfifv the 
tdlnUs nrednl for vntioii s nnerts, to ilvtemine the elfeetiveness of vtirious 
t)pes (}f eiluntfion in tlrvelnpinir these talents, and to formtdate the best 
/ttiurdnrei for nsastin;^ indieidnals in deflnittf;; their roles and phoning for 
the mnst eff'nttve and s<iti\fying use ( f their tidents, 
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Uolalril MullirH rarrinl nut during the paHl ihirly years have identified 
i rrlain lyprs of hrhavior measures as e»»onlial fur efTrrlive performance 
in spi^ihr H«»wcver, sysirnmtic and comprrhrnsivc studies have 
hern conlinrd primarily lo certain jt»l» areas in the military services. 
Kven Irss hus heen done in evaluating the comparative value of various 
types of edurati(Mial experience fc>r effective performance in particular 
careers. Uasl of all is knc)wn alxjut which counseling and guidance 
proccflurcs are most effective in assisting students to develop a realistic 
srlf cunccpl and plans for attaining their goals. Although many types of 
research stndit»s rM\ he rxpecled to eoiilrihule lo kiuMvlcdge regarding 
these malliTs, il is helievcd that most progress ( an h? expected from 
a comprehensive, large scale, long-range project using dectrcmic scoring 
aiitl «lata processing techniques. The planning phase for such a study has 
heeii nearly completed and partial support for the main study has now 
l)ecn made availahlc. As a preliminary indication of what such a study 
may reveal, some results on a relatively small follow-up study on high 
school seniors in Pitlshurgh are reported here. 

oil the hasis of a follow-up of 1016 persons five years after they were 
lc>le(l as seniors in high scdmol, it was found that 329 of this group 
cnltMcd eollcfj;e. Of this gr(»up, 193 had received a bachelor's degree from 
a college at the lime of the follow-up. About 30 percent (95) had quit 
<Mdlege liefore completing their courses. The others were still enrolled 
and expe< le<l to coinplele their courses. Ahout 60 percent of those enter- 
ing college graduated or expected to graduate in the course they had 
entered, (inly ahoul 2.1 percent of those going on io college took the 
course and entered ifie occupation planned on while in high school. 

Many those ilropping out (ir changing courses would have been 
adviscii not to cnlcr these courses on the basis of their pattern of apti- 
tmles alone. In a few cases their combined aptitude score for the specific 
course was a^ iniieh as I .S standard deviations below the minimum recom^ 
nieiided. Such wasted I'lTort is all too fmpient under the present system. 

7. Drrisinns nmrmihip which stmlmts should (tttaul college, and uhich 
rnllrge thry should ottrnd tO ntfoiu Omr ohjrctnrs slwuld he furivrd at 
hy a [noeessf>f sun rssive apf>mxintotions. Although tlie necessary data on 
which to luise these decisions is only partially available, colleges and 
students nni-t coiilinne tn make such decisions. It therefore seems most 
appropriate that both the college and ihe student begin early with a 
tentative set of dei isions and cohliiine to obtain as niiicli relevant data 
,H possible until the (inal deeisions regarding the student, his college, 
and bis mntscs must be made, 

li>r eN;unpb\ it seems appropriate that as early as the 9th gra<le the 
slndenl, with tli'* help ni the e(»nnselor, should begin lo hsl pos .ible 
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ram r rluMrt'H uimI possihir collrgt H and oournrs which woiiM hn appro- 
priulr for him. Collfgr aJmission?* ofTu rrs rmild providr ihr roiinsdorB 
of 9lh grdc- .slu<K»nts witli iiiforinnlioii regarding Iri.lalivo standards, not 
only with rrsprci lu.apliludi' and arhievcmcnl, but uk) wiih respect to 
fKTsoiiality, interest, m(»tivation, and aetivily factors. Each year during 
the sotondary sehool period, the possihilities should hf^ reviewed and 
the student should set certain goals for improving his information 
regarding hi^^ o vn ti|»litudes, interests, motivations, and other personal 
charueteristies as related to college requirements and college opportunities. 

It is helii ved that su< h a procedure would result in much sounder 
decisions than urc made at the preseni time on the part of both the. 
studtMit and the college. 

At a recent conference on testing for guidance, many of the test 
experts preseni favored delaying the administration of tests to determine 
specific aptitudes for career choices until the 12lh gradoi The altitude 
seemed to l»e, "If you can't provide precise predictive data which has 
heen fully validate d, delay doing anything as long as possilile." The point 
id' view of this discussion is, "If the tests and procedures are deficient, 
>larl as early as pi.)ssil>Ie and supplement and check on the findings for a 
parlirular student hy all availahle means/' 



Siiinniury 

What, t!ien, is proposed as an immediate program for testing for collegp 
admission and srholarship jjrograms? 

1. Develop working statements of objectives In terms of the desirable 
l)ehavif»rs of aflulls for colleges and iniiversities. These^ should be 
fof •used on nil aspects of life both in and after college. 

2. In the absence of empiri< al follow up data, use these working state- 
inenls in terms of luhavi<jr for establi.diing policies regarding the 
specific aptitude, personality, and interest patterns desired in the 
>tndenls admitted io a crdlcge. 

.H. At the same time, initiate long range follow-up research programs to 
pic»vide a lue^is f<ir confirming or revising the tentatively established 
admissions and educational programs. 

4. Develop tests to assist eounsel(»rs in helping students formulate 
realistic sclf-concepts which have clear implications fof college 
training and which will also assist admissions officers in deciding 
which applicants will contribute most to the joint program of at- 
tutning the objectives of the institution and of tht individual. 
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Very frequently the moHt crucial as well as the most difficult step in 
solving a complex pr(>bli»m is that of defining the problem itself, or of 
daiifying the issues invidved. I Ihdieve this is particularly true in 
relation to the prohhm of how to improve scholarship and college 
entrance examinations. It seems to me that the principal reason we have 
not produced more satisfac tory and useful examinations of these types 
in the past is that we haVe approached the task with loo narrow a concept 
of the problem to he ssidved, or with too limited a notion of the purposes 
such examinations should serve. Accordingly, I propose to spend most 
of my time here tiulay, not in arguing the relative merits of different 
types of tests in relation to different purposes, but in attempting to 
define and clarify these purposes; and especially, in attempting to make 
more clear the general nature of the problem as a whole. 

1 propose, further, to limit my part in this discussion to college 
entrance and scholarship qualifying examinations that are appropriate 
f(»r use in very wide st ale cooperutlve testing programs— programs such 
as the College Board (»r the National Merit Scholarship Qualifying Test- 
ing Programs— programs that are intended to serve a large number and 
wide variety of cnll(*giute institutions and scholarship donors, as well 
as a highly heterogeneous and broadly inclusive population of candidates; 
and programs, also, in which the tests are to be administered early 
enough to give tlic candidate ample time to make his major decisions 
and to compli te detailed arrangements for college attendance after 
the examination results are known. The latter means that the tests must 
be admini8i.:'/d while the typical candidate is still in high school, and 
hence that the tosts must be given by, or with the consent and approval 
of, the In^h scho(d authorities. 

It is fairly evident that nearly nil college entrance and scholarship 
qualifying tci ting must he done through wide-scale cooperative programs 
at the high schocd level. It is also quite apparent that there should be only 
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a Hjnall iiunihrr nf such |>ro^riims. Unltn liinalt'ly, this is iH»t now ihc 
rusr. I (In nut know liow inuiiy diirt^n nl prognims of tins lypr i\rv now 
coiulucWil unnuiilly in th«» high S( lu)<»la of this < (}UMlry, hnt I do 
know ihal iht' inimhn i^ far hn>;rr than viu\ possih'y Uv jnstifird, II is 
vrry ilonhlful that ihr nhhvi<hiul rolh-j^rs and a^Mu irs arr rrally any 
lu'llrr HiTvnl hy nianv pr«»^ranis than ihry conhl hr hy a wry 

xrnwU snialltT nnnih< r pr(>|M'ily phnuu d on a'^ VooiMTalive hasis. This 
larl is p'niTally rtciJ^riizciL and as a rrsnll many hif^h school principals 
arc on lh«^ vcr^c of opfii rchclhon at w!ial they "ij^htly regard as the 
nnrrasonahir dcniainU made on th<ir time and that of llicir puf)ilH 
hy this mnlti[>hcity ol testing programs. This is a situation that almost 
hnt not quite calls for a natural monopoly. Some competition in the 
provision of test services of this kind is undouht(Mlly whojrsomr auh 
desirahle, hut, for ^ ry evident and coinpelling practical reasons, wide- 
spriMd (hiplieation j)f elVort and eonseipient waste of the time of high 
school students and sialV-. nnisl he avoidMl. 

In Older to keep this discns^ion within manageahle limits, I propose 
Inrther to ecujsider here ordy examinations that arv? eonecrned v/ith the 
inlelleclnal altrihntes of the candidates. Other instrnnu'nts or soureea 
of informatitm, snch as int(»rest and personality inventories, attitude 
scales, hiografdiieal iidormation hianks and school records, of course, 
occupy a very impi^rtant [dace in tln^ whole proet^ss of determining college 
admissions or selecting ^Jmlarship riM ipicnts, hut there is hardly t^nte 
to consider all of the>e in this short discussion. 

maji^r (>oint that I »i>h to n»ake in this pa[»er i.s that our task is 
fnndap cntally one of linding a type of lest that will ntit jnst serve a single 
well-delined pnr[)ose, hnl that will satisfy a fairly large i:nnd)er of divrrstt 
rcjpiirerncnts. That i^. the prohlem is one of hnilding a muUiplr purpose 
rather than a siuf-h* pnrpo>c test. Uefor*^ attempting to identify or dtTnu' 
these purposes and requirenicnt . indivitlnally, let us consider hriefly «ome 
of the factors in the total situation that call lor this nudtiplieity of 
pnrp^>^cs. 

Oin* (»f the miJst important of these I have already mentioned — the 
(Mactical necessity of acci>m|di>hing virtnidly all scholarship and college 
entrance testing thrtaigh a very sniall nnnd)er of wide-scale eoopereli^( 
programs at the high school level. The test results olttainiMl in mc\\ 
pri»granis nnist he useful ti» literally hinidreds of difler^Mit institutions and 
agencies, i^aeh fd whi(h will nnploy tin* results in somewhat difTerent 
ways- often in cjuite markedly dilfcrent ways — than most others. The 
results will le uscmI hy sonu* tnstitntii»ns. f'»r i\\anipli», to ''skim the 
ereatn*' ofl'the top of the ahility disfrihution in a population of candidates 
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that is alre ady liijilily m U sclrrh *!* In ollu r mm sHerlivc itisliuilicmfi, 
HUcli as slalr-mi|)|)orlt tl utiiv« rsili«'s ^^ilh ttmrr a{>|iliranlH lhan they can 
liaii<llr, the Manif tost results may hr uscmI to cxcIikIp studenls at thr 
nllirr (*n<i of (Ik* ahilily srale in (lit* cMilirc uiisrdrcl^Ml population of high 
school gr.hlualf ^. In ^nnw iii^titutitms tlo' results will he used to deter- 
mine r«'a«liiie>*s lor a l^romlly mllural liheral arts eurrieulum, or f<>r a 
program ot j^t'orral riluratioii. In still others the results will he expeeted 
(o prediet ^ueet^^s in a narrowly speriali/iMl and teehnieal enrrtrlduni, 
a^ in rolli jijes of en«5inefring and the ineehanieal arts. In still otfiers tfie 
same le^l re^ults will he n^ed to ^ehM•t ^Uidents Inr an elementary 
leaoher training; prof^ram, or for a pre medieal eourse, or for a course in 
hu.sin«»*is manap*m«'nl, nr for a sehool nf ^oeial welfare, and so on, 
I have thus far onlv -^njr^esied some of the pt^ssihle variations of what 
. niighl he ri'^rartled a^ th«' eeiitral oi u»osl ostensihie purpose of scholar- 
rhip and ( (dlefie entrane*' «*\aminations, that «»f sehM tinp among the 
l andiilates uti the [»asis of their inttdleetual ahility. It is also extremely 
important to reeo^mi/e that dilTerent types of tests may serve ihis eenlral 
purpose eipially w(dl. yet may dilfer radically in the t'onsequences of 
their u^e in N\i l(*-^eale pref:;rams^ or in their ineidiMituI and often unin- 
tende«l elTeel-. F<tr example, twft types of tests may f)()th yield scores 
tliat show the -^ame rf)rrelation with eollef^e (grades, hut one may 
exercise a reslrietive or ntherui-e niulesirahle inlhienee on the high 
>elnH>l I'nrrii^ulum and the other may not; om* may foster j;ood and the 
otiier hail attitudes towards eollef^e pr*'paration on the part of the 
ean»lidates; or ime may he susi*(*|>til)le l(» snperfieial eramming or may 
lead to had eoaehing praetiers and th<' other may not. In such instance?, 
the Vide elleets** of the tests may oft*Mi he tin' determining factor in 
lest sehMiif)n, and to provide for these si'le elTeets is equivalent to 
♦ipeeifyiiif^' additional purpos«'s f(»r the examination. 

It is also exlren)* ly important to re- >{^ni/e th.at scholarship ami college 
entrance cxaminati<»n^ may readily he eonslructcd sf) tliat, witlnml any 
appr«'eiahle sai rifice in their ahility lf> serve the so-eallfMl central purpose, 
ti.ev ean ^«TVe many «)tfier cfpially important educational purposes an 
well. It is quite jiiw^ihle. lor cxamph\ lt> provide a test hattery that will 
not oidv prediet rolh^ge success or det<Tmine r« arliin ss for college as 
well as any olh« r. hut that will alsj> hr« highly useful to higli school 
cc»unsclors in advi-inf* students on their erhicalional atul vocational 
car«'ers, or on tfwir ehoice of type rd* college, and that wHl be useful 
as well to liigh sr iniol teachers in adapting instruction to individual 
<liirerr'm «s, atnl to high sch(Jol arhninistratr)rs in evaluating the entire 
ediuational olfcring of the school. Likewise, tfie same test hattery might 
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Ik? unrfnl to llio rollrj^jr aulhuiilirs lor |)lu< rriu*nl purpost'ii, or for 
purposes of rounaeling and f;ui<lanco, or to hrip ibem belter define the 
rollege's task by more athMjualely deserilaug the status and needrt of 
their entering stud(*nt hody. 

IManner» of sueli testing programs should not only lerogaize these 
[tossihilities hut r^hnuld regard as one of tluMr most important respon- 
sihiUlie> that of thus extending the usefulnes.^ of their tests to tlic greatest 
|M)H>ihii* extent. It is partieularly important that the results ohtained in 
sriiolarship testing programs Ik* innnediately useful at the high school 
leveK The high sehools are naturally relurlant to dev(>le very much time 
to te:^ling programs that are rooilueled solely for and in the interests of 
the colleges themselves. If the high sehools are free to (du)ose between 
two Competing seliolarshi|> testing programs or services, they will un- 
doidaedly give their sU|»port If^ the one in which they find the test 
rrsults nu)st useful for lln-ir own immediate [mrpcises. 

Furthermore, if we* are t(» avoid the past mistake of conceiving too 
narrowly of the |)Urpo.^cs and requirements (»f testing programs of this 
character, vve nuisl |»hin ihe |)rograms in con uicration of the present 
lundanicntdl needs of the Amt'rican system <d' education as a whole. 
\Vf have h^n grrally cniicerned, re<'(Milly, with the so-called Russian 
challenge to American education. We have become much more keenly 
aware id ihi* ingenl iw^'d of r.ii>irig tin* level (»!' intellectual eomp(*tence, 
not only of our srienlisls. engineers, and technicians, hut of persons 
engaged in all ty|ie> of iulrllcclnal activities in our society. Unfortunately, 
tip- American |>uMic has been i'n<dnraged to believe that to meet this 
challenge, we have (ndy to send more students to college, especially 
more talented -^tuderus, and [larticuhirly int(^ science and engineering 
courst's. This has been takt^n to mean that increased scholarship spend- 
ing, both federal and private, |)lus higher salari^'^ind increased facilities 
for scieru'e and mathematics teaching in the public scluxds, is about all 
that is needed. ^ 

Thost* of iiMV national h adiMS who have* rncouragefl*jblic belief in 
this a|>|>arcnlly ipiick and i*a^y and hence highly popular sohnion are 
p(Tha|)s more [loiitii'ians than educators. As our educational h*a<h*rs have 
gcne*rally recogni/.cd, our real [uoblcm is nof ln»w to senil more students 
to college*. We* alre*ady have nntre* slude*rns in e'f.lle'ge* than (»ur preseal 
faeiliti«*s ami instnn'tional staffs v\ill [M*rmit us to haudb* pro|)erly, and 
emr prcse'iU |u•^lvi^ious for enir highly talcnleMl students are especially 
inade cpiate*. Our re^al neM*d is not enen le» <M nd more talented students to 
college*. In the (irsl |)lai'c, practically all the* regally highly talented 
students ar(* alii*ady thi'rc. Tin* so.calle*d talented students who are 
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sla\iiif{ a\say linfii rnllrgr Inr fmaurial rrasniis are nu)»lly fairly far 
(lt»mi aloDg tlir uhility st ale If \sv did sut rot'd in neiiding tu college 
iUv \v\y small proportion of : rally talnited studeiiLs not now in attends 
ancc, hut \sr\r tti do nothing nn»rr for thrm than wc arc now doing for 
othrr talrritfd studnit^ in grnrral, wc wonhi an a nation suiely not be 
apprrriahlv hrltcr ofl lhan \m) arc ut prrsmt, 

Our real nrrd, tlirii, \< not just to yi'ud u larfi;rr numher of students to 
I'olli't^r, l.iiriih d nr nthi'r\vi^«\ hut t(^ crnich anti iinprovo thdr educaltonal 
t'xpciirihM'h a! all h'VfU - rollri^r, high stdund, and (drmrntary Bchool. 
Our rral nrrd-i arr to iili nlity thr talrntctl pupils mindi more surely and 
nuit h t arlit i Inng fu-fotv tlicy gi^ to collego — and to provide more 
adtMpialrly al all ^tagrs hir thc^ further devrlopineut of their superior 
tc»h*nts, and to ^i\r thnn the needed ineenlives to make the most of 
thtv^e enrielied opportunities and (»f their suptTior abilities. According 
to nearls all f)h^er\er- of eonteinporary Russian education, one of the 
iuo>t fintHtan«ling dilTrreiu'es between the Hussian system and ours lies 
in the general attitude toward edtieation, in»t oidy .miong students in 
Nubian sehiiols Imt .unoiig the Hussian people in p ^;eral, and in the 
inlere-l sho\N n in and tin- elVorl expended (Ui sejf-eduei.lion. If we are to 
iiiee! the Itiis^ian ehallen^'e, we nuisf, annnig (►ther things, find more 
elleeli\. \n.i\^ ot niuiisali;ig our students, fKU'tieularly our niost talented 
student-, or ol indurini: theni \k\ work harder, both in and out of school, 
al ihe l;i-k f>r ^eir-iin[>roM nn'nt» 

\\ bile llie\ have nut uenerally done so in the past, wide-scale scholar 
shi|> and enllt i^e i-ntiaiiee testi/ig prograuKs ean make a significant con- 
tiihntioM to tbe-e i>a^ie educational tu'imIs. Hy providing appropriate 
\\\\v^ i>r e\ iMiiuatiMii^, the prtigranis ean give the students a concrete 
and iinine li.itelv eHeelive inecMtive to work haider at the job of getting 
ready lor i*i)lleu»'. serve tins pnr[)osc, tin* examinations must measure 
direetiv the stn»lenl*s rvu'linrss for college, i»r the extent tO which he is 
fui'fuirnt to \)vkA\\ by tin* college cxpi-rieiu'c. That is. they must nicasure 
a- direetiv a*^ po--ible hii alaliiy to perform exactly the same kinds of 
eoniplex la>k- thai he ^^ill havi' occasion to perfiurn in ct)llcg«* and in his 
• later intellet tnal aetivities in general. The examination should therefore 

etinsisi in large [lart of exercises reipiiring the student to interpret and 
to eyaluale erilitallv iln* same kinds (d* reading materials that he v/ill 
have t»eeasii>n \o r»M»l and stn<ly in college, ainh particularly, that will 
reipiire him to do I lie same kinds of ctmplex rvnscmin^ and probhm 
sp/r/HiT that he will haye to do later both in and out of school. 

If the exaniinatioi; is u> have the maxiinuin motivating value for fhe 
high sehool ^Indent, it must im|Mess upon him the fact that his chances 
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III bring u<lniitt(Ml to collrgis or i^f Ix iiif; jHanlnl a si liol:iislu(N Jrpfnd 
ritjl only on his "In ighlnrsf**' or "inlrlligriin '' or olhrr iiinalr (juulilirs or 
fm lois lor whirh hi? is not prrsonully rrsponsihlr, hut rvm mnr/» upon 
how hard hv has wor^rrf at ihr task ol p-llinj; frudy l«»r rolh'gr, hoth in 
hi^h srhool and in tlir yrars prrrrdin^.^ hi>;h srhool. Thi* (examination 
nin-.l make hini IVrl that lir has cnrnvil {\w \'\^\\\ to f;o to roHr^r by his 
own (dForls, not that he is mtith-d to roMr^r admission hccanso of hin 
innate ahilili(^s or a|aitndr^, rrj;aidh*ss ol what he has done in high 
•^rhool. in other wurds, thi' examination mnsl he regarded hy him as an 
(tthivvnnent test, or as a test of his aeipiired or fA'eeAY^ef/ ahihtic The 
tasks eonstituting the examination inusl therefore ohvioiisly eorrespond 
til reeogiii/.ed high si ho^.l h-arning exjX'rii'nees, which means that the 
test exereises shouhl |>erhaps he grouped aeeording to major ureas of higli 
srhool in^trnetion- - the sin ial ntndies, the natioal seienees, lh(» human* 
ities, the eominniiieation skills, and mathematies. 

In thus (/rgani/ing the tests, however, the program planners must avoid 
jfny appearance attempting to dietntt* ihe content or th^ organization 
<d the high Sehoot rurrienhim. The colleges most not again lay lhemselv(*s 
op«-)i to the ehjpge iH^f tminating the higli schools through the eoll<«ge 
entwnee ti'slifig progij^nn, as they did decades ago under the old 
Itegents^ examinaliofT^system in New York or under the old (lollege 
Hoard sy^tenu The lest hattery rtiay therefore not consist of content or 
.s>ihieet matter examinations, corresponding to estahlished suhjects in 
(V tii^l^ sclui(J eiirricolum. At the sann^ time, tln^ importance (rf the 
student's kntiujnlirr musl not he neglecti il, 

These reipiirenients can 1»e met if the exariiination is eofieerned 
dikcrlly >Ni!h thr development of giSivrali/ed intellectual skills and 
ahihties, or with what the student can do wiph what he has h»arned, ami 
if if i-^ conccrncij on|v indirectly with w/yif he, has lenrneil, in the sens'* 
of s|)ecili«' items of inlormatioti or hits of knowledge. The necessary 
empl^H'>is on the seof^c and tpiality of the shnh'nl's knowledge can he 
'-eenfet]'>4^U' exertfi^es are ha>cll upon t'sl situations that emphasize 
dilh^cnces ;uhong tlu' eamliilatcs in their grneral informational or 
iiteatiiwtal luu'kgronniN, and in their [irevious educational exp(M*icnces» 
That is, the <'\vr)*i>r^/^V:.iuld givr* a tlrlinile advantage to the student who 
is already hest ihfrji;jrd /// ^rtuntl ahout the (indilcm io he stilved, 
or most expt'riern nVs^^n the soluli<Mi of such ^Jtohlems. This can and 
hlu»nld he diMie w ilhouY^coali/iiug unduly any cxanurier who happens 
nnt to |)«)s>e^H J) jiarlicular s^'^lic hit of inff>rmation. 

If thr e xaminations are surh that they provid^^eonereie and nnuming- 
ful inci»ntive:i to the individual higli>chool slndt r»t, tlu'V will iuTesftaril](, 
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ami Miv.aiM, ^vtw u ?^iinilar purptisf for \\\v inilivi<iuul srhool an wi^lL 
KxaiiiiiialiuiH tliut iiiriisurt' llit* rxinil tn whirh {\w iiulivnluul HludenlH 
urr iHopurt'il Inr i nllt-fic imi^nl oKvitmsly iit^lit al. aUu luiw wril the schools 
have preparrfl thnii for « i»lh»p\ Thi* ri^hl kind :»f Irsls >vill therefore 
iiiakr ilii* hi^h M^honls muw kn nly awaif ol* llirir own reHpuiiHibilitieb 
anil hhoiifomiu^H, ami al \\\r siiHr liiiif will f^ivr ihrni pimilive aid in 
m«'r:iiif^ llu'se rrHpiMisihiliiirs. hy ilrawin^ iUnv attrntion lo nroad areas 
i»r asprflM nl" ut hirM-riiiMil inti>l in iirtd of improvrmniL I need hardly 
pniiit out ihul rnlUm- riilrayur r\«n)inati<»riH of \\\v ly[K? generally 
rt»par<lnl a- iiiU lhfrriirt* or s.^hola^lir apliludf l<»sls, or differential 
apliliide li»sls. are aln»i»sl uhtdly us« h -^s for ihrsr purposes, as they are 
lor niolivatinf^ (Ik* individual stud(Mit. 

Tu appreeiah" iully whal kinil <d a prohleri. wr an» liere nmsidering, we 
niUMt givi»Honie allenlion In ••idl ouf more };eneial purpoi^e or requirement 
of eollege erilianer anil Rr}ndar.Hhi|) t»xaminali<»ns. It seems to me that 
onr l!u» most signilirani nlf^f-rvalitnis that one ran make eoncerning 
hurh exnminalioim \^ that lln-y, morr than any niher single thing, 
t nnslilulr the real ans>Nrr to the ipieslion, *'Who may gn to college?'' 
To the high srhool student who asks. ''What mujil I l»e like, or what muni 
I \)v al>le to do to hr ad:nilh'd lo eollegt*?'" the rralistir answer must he: 
' Diffrrrnl ro'lfgrs li:ivr many dilfen-nl rr<|uirements, Imt there is one 
thing that m-arly all id" them require in e(»nunon, there is one thing 
id>nnt wlvirli )ou may he rt-rlain— you must he ahle to pass the efUranix* 
exannnalions; that is. you iruist Im» ahir to do the kind of things railed 
Inr hv tin* fxamiiiatitui questions/* 

If wv are In look for the most universal and the really fiinvtiofwl 
fhiinttinti of tin* ilrsirahh* eollege student, or of his desired intelleetual 
jltril»ulf»s, wr nni-l look at the entranre examination. We should he ahle 
to lind tluTf a rfpr^smtiitive sample of preeisely the f^ame kinds of 
I oniplex tjwks tliiit tin* rnllege studrnt and infellertnal worker in general 
nnir » hr ahle to piTfnrm. \\V should expr' t to lind there a highly mean- 
inglnl defmilinn nf thr things that the high srhool and elementary seluud 
>hnidi| have pr» pared llu* student to do. I nerd hardly point out that, 
>irw(Ml as snrh 'Irlinitinns. most srliolarship and eollege entraut'e 
rxiUninaliniH usrd in the pas^ have hren utterly imidetpjate. 

Tlii"» issur i-i hn'i»ming partiridarly important with tlie iroTeased n>r 
t)l rntrjir>er i-Muninalions hy non-selertive in-^lilulions to det(»rmine 
who nuisi hr flrturtl admissinti hrrnuse ol the iiistilutinnV limited 
larilities. Ther<» is rral danger (hat surh institutions will plare undue 
and unrriliral relianre tm eiitranee exiunifiations '^imply heeausc sueh 
exiuninalions providr a drinonstrahly irnprrsonal and ohjeelive basis (or 
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innkiiiK uiipi)|Hilai ilci isioii)^ alHiul (lu* a|)pliranh. The plaiiiiei'H of 
(>iiil(*gt* riUrunt r (t iling |H'ogiaiii>> nuisl irni^iii/r lully llir roponHibiN 
ilit'.s that (h<'y (hu> a^MJinr, aipl nuisi inakr sprM ial rlTnrU lo provide in 
tfrtl fDrnt (hr hrsi pussihh* (Irliiiitioii of nhul is t'cally waiitrd in the 
iiirorniiig ^(udviit. ^ 

This h^atU lo my liiial gninal olisiM valitm nitirrriiiuK ihr nature of the 
prnhh'rn that wr arc Unr rDiiMtlrriiig. It is sinipl) that (hr pruhh'm i^ one 
whirh calU liiinlanp iiUilly loc a rutiotutl ralhri (hati foi an vmpiriad 
soltititMi, In th(* pd^t, (IrM^lopnitMilal Hork on srhdiarship and college 
enlraniM exanMnalions lias in ^<Mn*ial hern doniinulrd by the empirical 
or experinu iU: I i.p;troa<'h. I'he rnps if nu( (he >vhoh% of the examination 
haUery has u^Um!! • rMM^i^tcd oi tests oi the so-ralh^i seholasiir aplitudc 
type. In tioiul -uri'iij; -u<*h li-sls, \\r havi* heen ohsessed hy ihe ningle 
notitin th It thr i-^i^ tii ) rtMliet eoMege sueeess. Sinee we have had 
rr^adily avaihih i only ^e '^iiantitalive nuuisure of i^ueh ^^uecess^ the 
graihvpinnt averaps wv liav.- ali'iwed e>ur lest seleeiion procedures to be 
d<mnnai hy ihis dul)t•)U^ ulnioiL In many inAtanees, leHt!» and items 
have beeii ltd ainm^i ^'xelusivt^ly in terms of their correlations 
witli the^radc ; ,nt averaf;ts<^. t nrthermoie, and I think this is much more 
^if^niiit ant than has ht*en ^encially recognized, we have rharacteristically 
made np nw experimental try-out haltt^ies of very slwri suh-te»<t9. In thus 
doing, we have plact-d an nnduly Ingh premium upon the reliabiUty of 
the snhtt^sts, rather than npon their intrinsic validity. Our statistical 
tci hnii|ntv-» i>t test si^h-rlion have tended to ex<*lnde tests oi highly com* 
plex charat'trr - tests that in ri)ns«M|m»nee are almost inevitably low in 
reliability |M'r uint tt sting time, and that lor ibis rea^ion alone ^will 
in short lorms shou |o^^ correlations with the criterion, eveii though 
their intrinsic validitit .s are quite high. We seem to have been undnly 
coni erniMl \>itli the rffirifrti) of our prediction instruments. That is, 
\sv sei-m to hast* had as our tibjcclive that of siu^uring a liigh correlation 
uith the criterion in tin* 'hortest p(»ssible nnmunt of testing time, rather 
than that of attaining the highest possible validity in whatever amount 
ol time is needed to do the job right. 

W bat is nuiidi nn)re serion*^, however, is ibat in selecting or construct* 
ing the •Aob'tcsts to be tried out ex|)erim(Mitally lor use in such batteries, 
have strongly lasorcd tc^ts that are highly boniogiMn'ous in character 
antl" simple in stmetnre, tests that show a high correlation with the 
t riterion and low correlations with one another. That is, we have tended 
to exi lmle complex types of tests even from initial consideration, and 
often have not even tried them out in the experimental batteries. When 
we have im Iniled (hem. we have not matle them long enougli to compare 
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favnral)!)' in trfialnluy v.Uli ihr uIIht Irnh inVtl oul, uiid Iwiive hav^ 
i|i»nmnl ihfin tn r*litiiiiuili(iii lalrr. ni^viously, tlu» final ballfry of tests 
Ht'lninI raniKit hr any lit lli r than ilut IrnlH from which the sfleclion 
wa.H iiiailr. In oiir «'ffnrls to analyze coiiiplrx menial procesftes into 
simplr aiui iniir|M n(l« nt roinponrnts, wr have analyml out or otherwise 
rxrluilnl ihr musi im|»(»rlanl n»m[)onrnls (»f alU-lhc most essential and 
cliH|inj;niHlnng riiarant-rir^lir nf rat li of wliirfi I.h its very complexity. 

h is irnpi»rlaMl, ol rourM', llial ihrsiuilrni [M»s»esH muiiy^specific skills, 
as well an lhat \\r havf a iar^e sidrr nr>fMM ifi. information. What ift much 
mnrr important howt^vrr, i^ thai he Im ahir lo use all uf these skills and 
knov\lnlf;rs at ihr time and in the right etimhination with one 
anoiiitT in iht- sulntinn of hif;hly r(»mplrx prolilems. It is the complex 
or^ani/alion of. and ihr inl^ra^tion^* among, these skills as they art 
hrin^r usnl thai is mnnt iniportani, not jiHt the specific skills themselves. 
It ix rxtremrly iin|»urtanl, furlhrrmore, that Ihe^tuilent have acquired a 
HuniHl srnsr nf valur-i, and that he Im? aide to make decisions and lo reach 
major n>ni hisions in [»ru[»rr ronsiilnation of these values. In other 
words, it is nio^t iniporlant that lie rx< rriHe sound judgment in all that 
he dors. Thrsr ahiHtirs |o wv many s[HM*ifir skills at tlie same time and in 
Ihr rif;lit roinhinatioti, io wei^li vahies, to do complex reasoning, to 
rxtTrisf* jud^nn nt, and many ollitn- similar ahilities definitely cannot be 
ni*a>uird in tlir ahshai l or in isolalioii from one another. Certaihly 
ihcy arr not traits tliat air [^syi liolo^irally simple in slructurr. They can 
l»r inrasiirrd, hiil ordy ii) rrlalioii lo llir romfdex situations in which 
ihey arr drniandrd. 

Il is my ronf'iititui, thru, that even for the single purpose of predicting 
llir ipirsli<»naldr grailr-fManl average i rilrrion, tlie techniques that we 
have iieen ein()h>yinf' arr far from |»erf<'rt. As a means of selecting tests 
that will [jredirt niiilti|dr and roniplrx crit<Tia such as we should be 
tlrvrlo[)ing lo lake tlir plarr: of tin* giflde-point average, they are clearly 
mueh less adtMpialr. As a mrans of hrljHng us dr/tne major educational 
goals, ihrv arr iiltrily inadcquale. 

riir ialtrr. as I see il. is ihv real naliire uf ihv joli of improving college 
cntraiirr and srlinlnr'^lii(> trsting prngrams. I have suggested quite a 
inimhrr t.f diflrrriit rrqnirrinenls lliat I lirhevcsuch examinations should 
serve. Kvrii so, there arc many others that I have not c^ven had time to 
menlit»n. Il is ohvioiisly impoHsihIc lo eonsimct n single test battery that 
will ^erve all of fhese many purposes perfectly, or, for that matter, that 
will M-rve any om* of them to our c'onqdete satisfaction. It is possible, 
however, in provide a single Imttery that will prove highly useful in 
relation to evtMy one of these [uirposes. This can be done without 
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appHM iaMv hih l irn nin tlii' UHrliilni'SH ut llie liatlery Uii any single 
pm|inH*\ iiii liiilidf; lhal t»f i\u* \nri\'\r{U)t\ of HiHM'e.HS. IiKleni^ llierr in 
now [ilfrity ol rviiliMH r* that tfHts of llir kiini tliat I havf Hii^t'strd will 
<|i) a lifttrr nl [MrMlirtiiig < oll(*ge huimu^sh tliaii will any availalilt^ 
liall<»iifs n| irslH ul -^iM riric a|»liliHl<'S ur hkills. If wr art* to prnviilr ptnl 
U. roiitiunr tn iuiprovf kiiul of \vh\h nrnlnl, wr mu^*t nM(»gnize 
that llir til k i-^ rs-%rnlially our of <l«'st rihinj^ or fAym/n^, in tfrmn of lent 
situations, souu* of thr hroati nlurationul goals that the students and the 
^rliooU arc Mow trying; to attain, 

I l annot eni|»lia-i/.e to<» striuigly, howi'ver, that it ih delinitely not tlie 
funetion of pl.ninrin of srholarshi|i and collegr (Mitrance examination 
|Mogranis to o trrmine or to st^t any edurntion goals. Certainly it is not 
their provinre to alti inpt through the tests to briiif^ about vhnnges in the 
high s('h<*ol enri irnluni, no matter how desirahle. It is proper and desir- 
alihs however, that the tests aeenrately deserihe some of the hrnad 
edm ational goals which are alrt^ady universally aereptefl, and that they 
enipliasi/c tlir nerd for further development of generalized ahilities that 
are of sidf evident and utiqnestiotied importanee. In eonstrueting tests^ 
of this elunarter. some statistieal and em|)irieal teehniipn*Harc UHcful and 
ne< i->^arN, hut they are of sreomlary impoitaner. This is a task that calls 
Inndatnentallv hir great ^kilL imagination^ and ingenuity on the part 
of the item wriln , rather than ft>r skill in statij^tieal nuuiiptdaticm. The 
wiiting <d iteip-^ for ^\u'\\ \v'<\> rails UiV the very highest level of talent or 
rontpetetu e avaiLd>le in tlie uhoir held of ediieational measurement. 1*he 
wlnde ta>k <d' tt*>t eon>trnrtion is one that ealls for a logieal rather than 
lor an empirii al appi oaeh or one that, mt»st of all, demands the exereise 
of sound judgment. 

In elosing. I N\ould like to repeat and emphasize a point that I made 
earlier that it has heeii niy purpose hrrt* to argue the nicritft of any 
partirular t«*Hl or testing programs. My eoneer ii here i^ only with the gen- 
«Mal diiertifOi in whieh our further efforts at test /m/^roeeff/eri/ should he 
pooMt il. Wr \\H\r made gri*at progress in the pa^t »ti devrloping tests of 
the ty(>e 'luggr-lrd, Imt rrrtainly there plenty of rot>m for and great 
nrrd lor ronlimird ifiqu'iurinetrf in all presrnt in Imments. I am e ui-^ 
\m« f il llial. il \\r a|»pMKO'h thi*^ task id test improvenu'iit in the manner 
Huggrstrd, if we re« ognize that seludarship and eollrge entram e rxam'» 
inations <'an and shmdd serve a wide variety of purposes, ami if we 
re< ogni/r tliat our task is fuudanuMitnlly oiu* of defming the grneralixed 
and ( ninplex ahilitic s we want the liigli nehool stude nt graduate to have 
developed, wr ran huihl intfi sueli programs ^lune really signifieaiit 
posiiivr Vidues for Amerii*au edueation. 

li:< 
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What Kinds of Tests for College 
Admission and Scholarship Programs? 

AlexanDfr G. Wesman, Associate Director, Test Division, The 
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The It'sls wo nerd for college admissions purposes arc those which arc 
rcliahle, eflirienl, inexpensive, confidential, comprehensive, unique, 
reflei live of the curriculum, independent of the curriculum, fair to late 
developers, and valid for f very curriculum in every institution of higher 
learning. Unfortunately, no surh set «)f tests exists. In fact, no such set 
of tests can exist The deman Is t»f earh instituticm are, or should be, 
unique the tests one college needs will necessarily differ in some ways 
from the tests other institution:! need. Any attempt to specify the same 
testing program fj)r all colleges, or all srholarship purposes, is inherently 
self-defeating. 

The central issu* in ehoosing tests for college admissions purposes is 
the same as for any other purposes— what use is to he made of the test 
results? This in turn depends on the nature of the individual institution 
its goals, its role in our society, its facilities, its philosophy. The grow- 
ing prevalence of national and statewide programs embodies a real 
danger that individual differences among institutions of higher education 
will be overlo(»ked. The advantages of uniform tr»iing programs may be 
purchased al the excessive price of ignoring one of the greatest strengths 
of our educational system—the variety of functions performed by our 
collrges and universities. If every institution were concerned with select* 
ing for admission only the intellectually elite and in providing the same 
kind of education to all those it admitted, a single set of tests might be 
prescribi'd for all. As long as we have state universities and highly 
selective private colleges, liberal arts colleges and agricultural colleges, 
cultural emphases and vocatiotial emphases^ it is unlikely that one set of 
tests, or one program of tests however thoughtfully devised, will ade- 
quately serve the needs of all. Rather a variety of tests and a variety of 
programs is essential if each institutiot) is to approximate the require- 
ments of Its own special circumstances. 

Ortainly every institution needs tests which are reliable; but is the 
^ame test reliable for every institution? Not iirdess it is a most inefficieht 
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iiiHtnitiifiK. (Uir iiiHiituliuiis of liiglirr ctlutuliDii vary witlrly with re* 
H|)cf-t to tlir IrvcU ol lulcnt in the studfiitm tlu7 admit. If a test iH to have 
enough (iiflicult material to dii^criminate reiialily among the top twenty 
per cent of our studentB, and enough easy material to Jiscriminute 
reliably among the hottom twenty per cent of college freshmeni the test 
will he far too hmg for practical use in either group. Further, the large 
^ numher of eany itemH ifl likely to horr the hetter students almost as much 
an the large numher of diflicuU items frustrates the less ahle; and, in 
each case, the Hciiirns of inappropriate difliculty represent inrflicient 
mcasuremrnl. 

An exumpic may focus the proMcm of range of talent more sharply. 
In one stale university system last year, the same set of tests was used in 
all the units of the system. The average verbal test score of the entering 
freshmen in one of the institutions (College A) was more than two 
standard deviations below the average score of those entering another 
institution (Clollrge H) in the same system. College D, whose freshmen 
scored highest in the state system, ^as only average among all the 
schools which use this test. This uggests that differences in mean scores 
bdween the most selective schools using this test and low-scoring 
College A are fully four to five standard deviations. It is^robable that 
for a majority of the students in College A the later half of the test 
provided a depressing experience, but.no real measurement. For these 
examinees the effective portion of the test was composed of perhaps 
half the items printed in the booklet. The reliability of the test for 
discriniinatinf; nniong these students, must be assumed to have suffered 
accordingly. 

Tin* tests shouhi be efbcient— they should occupy as little of the 
school's and the student's time as is necessary. This is not to say that the 
lime spent in testing is not as well spent as if equal time were devoted to 
other kinds of expeiience to which a student might be exposed. Hather, 
it suggests that effu ient tests permit the gathering of more information 
within reasonable time limits. If four hours arc to be devoted to testing, 
wr should seek full value for those four hours. There are programs which 
compel some students to stay overnight in an out-of-town lodging; if 
more efficient testing can eliminate this burden, such programs should 
be matle more eflicienl. The student may not be in a position to protest; 
but the captive stale of the student should not make his captors Itjs 
luerciful. 

To be mo!«l i^fTcclive, tests should supplement information which is 
( therwisc avail;d)h% rather than duplicate such information. The college 
which draw s its students from a small number of local secondary schools 
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hIukiIJ hv al)le to acMTpl the Hludonls' high schnol records as evidence of 
their at adnnic preparation; achievemeiil tesln are of secondary utility if 
they art* devotetl to assessing the same information aH is represented hy 
well-iiiu|iTsto(Hl srhool grades. Where there is eonsiderahle div^^rsity 
among frrdrr srho!»ls in tht ir t urrirula and graihng standards, the use of 
at hit'vt ment tt sln may Im» ninre tlrlmsiMr. lloHt ver, in «>ur enthusiasm 
(or tests Hr shouM not l«)rgi'l what research has so often demonstrated— 
that even Y,\u rr students are drawn from diverse scrondary schools, 
high HvhiuA avrrage is often one oi the hest prt^dietors of performance in 
rtiHrge. At < i^nhngly, instruments whirh are h-ss directly reflections 
the suhjert matter rompt»tente of the student may provide more new 
infiMnuititiii t onrerning him tlian do tt^sts in suhject matter for which I 
gr'mlt^s arr ahfady availai>le. | 

Thert' an- at least two other atlvuntages to the use of non-curricular 
t<*sts. Thr use n( arhii vrment tests (m eollcge admissions all too often 
♦ xerts a ilis[Mt>pt>rtitMiatr influmre on the srcondary school curriculum 
and the set tuitlary selu)ol teacher. A('hievement tests are valid if they 
measure what the st luM^I wants to leach; hut sriiools frequently behave 
as though their teaching is valid if their sliidrnts do well on some es- 
tremetl achievt nu^nt test. Stmir years ago it was commonplace in New 
York {n hear the complaint that the final srmester of a course was de- 
voted entirrly t(» spct ihr preparation of the students for the Regents 
examination; then, for several years, the issue appeared lo have been 
n solvtMl. Today* t)nrr again, another .M^t of tests occupies a similarly 
iloniinant pi»siiion. Tlufsr of us wlu» are respimsihle for developing such 
tests may hnd ready n lup- in the statement that ice do not recommend 
that st fiool oi irai lu r adopt this subservient role — that the tests are 
inttMitlrd to follow, not (b termim^ the curriculum. Rut as long as subject 
matttT tests svrvc t olK gr admissions purposes, wc must expect teachers 
whi> are anxious to help eollrge-hound students— and teachers whose 
own perft^rnuuh r will bt- jurlgcd by thrir pupils* success on these tests— 
to t'iMieenlrate on the tests as nnieh as on the course. 

A second advantage t)f tests which arc not curriculum oriented is their 
potential (or rescue (unctions. There are students whose formal a^a- 
tiemie prt paraliou is tlefet tivi*- tlnme wlu» were unstimulated by their 
t'onrses or their teaehers- wh<> may nevertheless be salvaged. Their pre- 
vious failure to b»arn may have been the result of delayed maturity on 
their [mrt, t>r of an miitispired etiucatimial envircmment. That these 
^tmlents have not learned nhat their course's riffercd would be dt»cu- 
meiitcd by course grades and by achievenient tt^sts alike. To reveal that 
they rnulfl b ai n reipnrrs a different kind of predictive measure. 
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Tlir pn»|>(tition n\' siutlmi^ do jHioily in hi^h Hrhonl, ihen IiimI 
ihrniM'lvr.s \slirii iln y .irr f^ivi M a rhaiuT t(» do t'oHrge Work, may hv 
sfnall, Lul tin- id.v<.|ulr luirid t r of siu h -Jludruis is large enougli lt» 
warrant Mfii»u> attrritinn. A larj^r mitlweslrin slalt* universily found ni 
its l%7 t-nlrring t laf-s IHH sludt ills \>ho scorrd among ihe lop Iwriily. 
U\v |)rr tfnl un lln' Collr^v (Junlijivation Tvsts, and witc in ihe I«»wrsl 
quarlrr ol llirir high x hnol chi^s. M»»rr llian half of ihese slndcnls al- 
lainni a lirsl MMnr Irr ^ladr pninl avcragr t)f 2.0 (li) or l)rller. In ihis 
^auH' rla>s, ihcrr wvvv :\\ I tVt-^hini*n who wrrr also in the lopquarlrr on 
ihr (;(/r ;nid in ll)r ihird (juarltT in high m Iumi! rank. Thrce-ftjurlhs 
iht M* ficshnirn rurnrd a-giaJr poinl avcragr td' 2.0 or hotter. Tliese arr 
^hhh'nls ;i(U) ol ihfni in a frrshinen class for whom |)r"»;n(»sis 

on ihf hasis of pa>l m a Irniir arhirvrmrnl wonid he peRsimiRlin, hnl who 
wcir rorrtM'lly idrnlilir(] hy ihe trsl.s as heing capulde of at least initially 
sali^farlory work in < ol|rgr. 

llMMt' are a nurnlu r of considerations which eatdi institution must 
rrsoKc lor itsell hrlorr it adopts an admissions hattory or accepts a 
jMoi^rani dfviscd hy sonu* out-iifc ageney. 

I. Will a policy of srhs^ive atlmissioiiH he practiced, or is the in- 
-litnliuii ohiigrd. [jcrhaps hy state charter, to admit all applicants 
who ih( ct certain inininnmi re<]uiremenls? 

A hi;ihl\ c .1 I'Micd private lihrral arts college has the privilege of select- 
ing onl\ iliu^e stndcnl^ who show greatest intellectual promise. Some 
^lat^ nrii\ri-.iiii> do not liave thai j>rivilege. Society has prescrihed 
thai n«>t onlv the ehle shall he educated; all who can profit from ctjilegiate 
education in diverse enrrienia anti at varying levels of intellectual demand 
are to he given ti.e oppoitnnity lor fnrther acatlemic training. It is true 
lhal e\i'ti pnhlirly snp(»ortip| in>titulitnis are hnding it necessary, he- 
ean^e o| ^^wellirjg hordes of ap|>lieants and limited classroom capacities, 
to rxrrrise snnie seleeliun. Ihit llie exclusion of the least promising frt)m 
thegrrat nias- of applicanN i^ a «piite different task from that of choosing 
a siujII rnnnl'cr the rlile IVtMu an already self-sclected grtnip of lop- 
ranking candidalr^. One shouhl expect that tests differing in difTiculty, 
and [pcrhap-^ in kind, are neei\*^sary for these dilTerent tasks. 

L*. W helher or not ^eh'cli\c adniissitnis wilLhe practiced, are students 
php ed in difleretil elasscK or sections on the l)asi« of lest results? 

II Ire-lniian eont ses aie ofTercfl al niore than one level to students with 
dillereni aeadeniic |)reparation, ai hievcment tests may he useful to ap- 
praise the siihleni's eonipetent c at entrance. If there is a ct)Urse in chem- 
istry l«»r ail.aneed shidenls and anotlu*r ft)r students who have not 
pre>ion-lv taken chemistry and (/ ihf student's r#»eord of high 3rht)ol 
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viiur^vH is jiKlf^t il iiisiiffit it*!!! l(» irslify t(» \m knowledge of the subjocl — 
a rlirniistry ttvst may hv advisable. If, on the otiier hand, the same firnt 
rourae in olTtTnt (o all students regardless of previous exposure, a sub* 
jeet matter test i^; probaMy le»s crucial. 

Will the faeiihy make u»e of the test results in its teaching, or are 
the tests (uiinarily to serve a screening function? ^ 

li tlir hinlu^y or liistory teacher will use the information gained fioni 
trr>ts in his suhjtMi riuitter to plan his instruction, achievement tests may 
Uv rsirahle fur stutlenti^ entering the course. There is many a faculty 
iiirniher who, rightly or wrongly, , expresses indifference to how much 
sulijct t matter the stndent Sas learned before he enters the class; rather, 
it is nlu tlier the student laii learn, and is willing to learn, what the 
IM'iifessor wishes ti> t^^aeh him that is crucial. This teacher may be one 
wlio is unc(»nvinced by tlie suggestion that what the student has learned 
ill the past is predictive of what the student will learn in the future. Or, 
this may lie a tearhfM* wlio, faced liy overflowing classes of students with 
a uide range of previous preparation in the subject, has recognized the 
futility (if trying to Uiilor his teaching to the varying amounts of knowl- 
edge possessed by the indivi(Uial entering freshmen. All students are 
(rratf d by these two pr(»fessors as essentially equal and uniformed on 
rittraiire to the class. High scho('l course records in tliesul^ject are not 
mmmi as helpful liy tliese professors; achievement te80re8ult8 are likely to 
lir efpially Ignored by them. 

If a t(Nt is to Ite hsed primarily as a screening device rather than as a 
for instrurtion, a scholastic aptitude measure will be more eflicient 
;in«l more broadly appliealile than a subject matter test. 

l. lb)W many rnri ienia does the school offer? 

riie small lil)eral arts college may require that all students take^a 
•standard, presn'il>rd rnrrirulnm for the most part, with a small number 
i)| elei tives. A lar^^e ntate university may be composed of a number of 
r()llrg(\s sneli as srieiiee and letters, agriculture, education, nursing, 
pluumat y and enf;inrrM ing with eacli college in turn offering more than 
one cnrrieuluin. Tlie variety of curricula in the state university probably 
assni'(*s that \Ur siudi iit l»odieH (»f the several c(dleges also vary, in level 
of ability a^ well a^ in areas of academic interest. A minimal program 
which would proNc satisfactory for the homogeneous freshman class of 
the liberal arts college might mvW prove inadequate for the heterogeneous 
population which enters tlie complex state university. 

Dr. Harold CulliksiMi. discussing several papers at a recent symposium 
said, in e(T(M t, ''Onn* again we have heard excellent presentations of the 
problems in area and the questions that need to he asked. It would 
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be nice if, eometime, wc might have a paper whirh nuppliod ani^wers.'* 
SincD the views embodied above expresH the conviction that no single 
battery of tests wilt serve equally in all institutions, an attempt to pro- 
pose such a battery would be inconsistent. NonetbeJrss, a general ap- 
proach can be presented. 

The core i)f an admissions testing program should include measures 
which appraise the student s command of our two most important symbol 
systems, verbal and quantitative. The ability to manipulate verbal and 
numerical cpncepts has almost invariably been shown to be associated 
with success in future learning at all educational levels. Opinions differ 
as to how these abilities may best be tapped — by synonyms, antonyms or 
verbal analogies, by numl)er sericH, problem-solving, or numerical com- 
putation— liut there should be lillle dispute that some kind of effective 
appraisal of ve*rbal and numerical al)ilities is essential. A third component 
might be a brief test of information, sampling broadly from the general 
areas of physical and social science. This test would be intended to pro- 
vide some reflection of the student's educational background where 
feeder schools and their marking systems are diverse. As a fourth com- 
ponent, a reading test might well be included as much for use in guidance 
and identification of students i[i need of remedial training as for pre- 
dictive purposes. Then, because tlie student's outlook toward s(:i>ool may 
indicate how he will react to the (Educational process, a survey of his 
beliefs and attitudes with respect to study, to teachers and to the general 
academic envimnmcnt might well be in order. Beyond this core, addi^ 
tional testing with respect to special abilities (such as space perception) 
or specific subject mattci competence (e.g., formal mathematics), may be 
added according to the particular character of thi* institution and the 
readiness of faculty to utilize the test results as a basis for teaching. 

With respect to tests as a basis for awarding scholarships, one needs 
first to inquire what purpose the scholarships are to serve. If we are 
simply seeking the academically most promising, then a verbal and 
numerical test of sufli(Ment difhculty to challenge the top two, or five, or 
ten per cent of secondary school graduates will do a satisfactory job. If 
scholarships are intended to provide additional recruits for special 
areas of our society— -scientists, social workers, teachers or missionaries 
—the tests to be used, and scores which will qualify the accepted 
candidates, must be tailored to the task. 

One could make a brief for examining the subject area competence of 
students in areas in which they had not had previous schooling. There 
are undoubtedly students who have learned a great deal about mechanicol 
things through their own curiosity, through recreation and cxperi- 
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menlMiou, through observation and self-tlirrctell reading rather than 
through formal course work. If scholarshipft enrouraged the further 
development of students such as these, a potential adtlitional source 
of creative talent might be uncovered. A brief oouhl he mode, too, for 
awarding scholarships to those ^vho are not in the highest ranks of 
academic promise but who can contribute importantly nonetheless, Wc 
award srholiirships to students many of whom wfiuld po on to college in 
any \ event. If in-^lrad, potential trurhers could be located and sub- 
sidised -students who are not in the lop ten per cent of their class, whose 
scores on our usual schnhuship tests arc medittcre, who would not 
otherwise pursue further education, who might nut earn the highest 
grades in a teacher training course but who could successfully negotiate 
a teachers college program and would enjoy teaching— more would 
be contributed by scholarships for this purpose than by the ego-satisfying 
but unessential suppr-rt of those whose careers are not genuinely affected. 
Lower qualifying scores, or even different examinations, should b©r 
employed for this kind of scholarship award. 

To summarize: the kinds of tests that are appropriate for college 
admission and scholarship programs are those which are best suited to 
the individual institution and the particular purposes of the scholarship 
tloiior. No ont testing progfam will suit all schools or all purposes. 
There are many good tests. It is incumbent on the conscientious user to 
selert from among them those which most nearly meet his special needs 
and circuinstancrs. Otherwise, the tests which have provided milestones 
along the road of educational progress may become millstones around 
- the neck of the educational process. 
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